-- Logs begin at Wed 2022-09-28 15:00:46 UTC, end at Wed 2022-09-28 16:01:46 UTC. -- Sep 28 15:27:04.295108 dv-cinder-859319-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 28 15:27:04.305167 dv-cinder-859319-2-iscsi devstack@keystone.service[81593]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 28 15:27:04.305876 dv-cinder-859319-2-iscsi devstack@keystone.service[81593]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Sep 28 15:27:04 2022] *** Sep 28 15:27:04.305876 dv-cinder-859319-2-iscsi devstack@keystone.service[81593]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 28 15:27:04.305876 dv-cinder-859319-2-iscsi devstack@keystone.service[81593]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 28 15:27:04.305876 dv-cinder-859319-2-iscsi devstack@keystone.service[81593]: nodename: dv-cinder-859319-2-iscsi Sep 28 15:27:04.305876 dv-cinder-859319-2-iscsi devstack@keystone.service[81593]: machine: x86_64 Sep 28 15:27:04.305876 dv-cinder-859319-2-iscsi devstack@keystone.service[81593]: clock source: unix Sep 28 15:27:04.305876 dv-cinder-859319-2-iscsi devstack@keystone.service[81593]: pcre jit disabled Sep 28 15:27:04.305876 dv-cinder-859319-2-iscsi devstack@keystone.service[81593]: detected number of CPU cores: 4 Sep 28 15:27:04.305876 dv-cinder-859319-2-iscsi devstack@keystone.service[81593]: current working directory: / Sep 28 15:27:04.305876 dv-cinder-859319-2-iscsi devstack@keystone.service[81593]: detected binary path: /usr/bin/uwsgi-core Sep 28 15:27:04.305876 dv-cinder-859319-2-iscsi devstack@keystone.service[81593]: your processes number limit is 47680 Sep 28 15:27:04.305876 dv-cinder-859319-2-iscsi devstack@keystone.service[81593]: your memory page size is 4096 bytes Sep 28 15:27:04.305876 dv-cinder-859319-2-iscsi devstack@keystone.service[81593]: detected max file descriptor number: 2048 Sep 28 15:27:04.305876 dv-cinder-859319-2-iscsi devstack@keystone.service[81593]: lock engine: pthread robust mutexes Sep 28 15:27:04.306572 dv-cinder-859319-2-iscsi devstack@keystone.service[81593]: thunder lock: enabled Sep 28 15:27:04.306572 dv-cinder-859319-2-iscsi devstack@keystone.service[81593]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 28 15:27:04.306572 dv-cinder-859319-2-iscsi devstack@keystone.service[81593]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 28 15:27:04.346148 dv-cinder-859319-2-iscsi devstack@keystone.service[81593]: Python main interpreter initialized at 0x55c7347f49c0 Sep 28 15:27:04.346148 dv-cinder-859319-2-iscsi devstack@keystone.service[81593]: python threads support enabled Sep 28 15:27:04.346148 dv-cinder-859319-2-iscsi devstack@keystone.service[81593]: your server socket listen backlog is limited to 100 connections Sep 28 15:27:04.346148 dv-cinder-859319-2-iscsi devstack@keystone.service[81593]: your mercy for graceful operations on workers is 90 seconds Sep 28 15:27:04.346532 dv-cinder-859319-2-iscsi devstack@keystone.service[81593]: mapped 403077 bytes (393 KB) for 2 cores Sep 28 15:27:04.346613 dv-cinder-859319-2-iscsi devstack@keystone.service[81593]: *** Operational MODE: preforking *** Sep 28 15:27:04.346837 dv-cinder-859319-2-iscsi devstack@keystone.service[81593]: *** uWSGI is running in multiple interpreter mode *** Sep 28 15:27:04.346837 dv-cinder-859319-2-iscsi devstack@keystone.service[81593]: spawned uWSGI master process (pid: 81593) Sep 28 15:27:04.347039 dv-cinder-859319-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 28 15:27:04.348109 dv-cinder-859319-2-iscsi devstack@keystone.service[81593]: spawned uWSGI worker 1 (pid: 81604, cores: 1) Sep 28 15:27:04.348109 dv-cinder-859319-2-iscsi devstack@keystone.service[81593]: spawned uWSGI worker 2 (pid: 81605, cores: 1) Sep 28 15:27:04.348109 dv-cinder-859319-2-iscsi devstack@keystone.service[81593]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 28 15:27:08.640986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 28 15:27:08.640986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: from cryptography.utils import int_from_bytes Sep 28 15:27:08.641784 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 28 15:27:08.641784 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: from cryptography.utils import int_from_bytes Sep 28 15:27:08.649013 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 28 15:27:08.649013 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: from cryptography.utils import int_from_bytes Sep 28 15:27:08.649481 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 28 15:27:08.649481 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: from cryptography.utils import int_from_bytes Sep 28 15:27:08.691086 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 28 15:27:08.691271 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 28 15:27:08.691459 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] command line args: [] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 28 15:27:08.691722 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 28 15:27:08.691898 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ================================================================================ {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 28 15:27:08.692151 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] admin_token = **** {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.692396 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] config_dir = [] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.692583 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.692762 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] config_source = [] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.692940 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] debug = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.693136 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.693622 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.693718 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.693930 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.694180 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.694408 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] list_limit = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.694586 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] log_config_append = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.694760 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.694984 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] log_dir = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.695111 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] log_file = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.695337 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.695567 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.695810 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.696053 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.696227 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.696400 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.696640 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.696749 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 28 15:27:08.696872 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 28 15:27:08.696995 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.697107 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] command line args: [] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 28 15:27:08.697226 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.697340 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 28 15:27:08.697452 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.697577 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ================================================================================ {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 28 15:27:08.697698 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.697812 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] admin_token = **** {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.697953 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.698068 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] config_dir = [] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.698186 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.698310 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.698425 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.698543 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] config_source = [] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.698657 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.698771 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] debug = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.698771 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.699026 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.699141 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.699741 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.699867 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] publish_errors = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.699984 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.700096 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.700096 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.700297 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.700398 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.700509 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] list_limit = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.700509 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] log_config_append = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.700707 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.700810 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.700917 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.701026 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] log_dir = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.701135 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] standard_threads = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.701238 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] log_file = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.701353 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.701460 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.701571 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.701673 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.701775 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.701900 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.702019 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] use_journal = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.702019 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] use_json = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.702240 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.702361 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] use_stderr = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.702484 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.702599 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] use_syslog = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.702712 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.702831 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.702945 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.703059 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.703180 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.703295 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.703428 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.703542 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.703663 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.703783 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.703897 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.704015 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.704130 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.704243 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.704364 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.704485 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.704599 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.704711 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.704711 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.704923 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.705089 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.705089 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.705306 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.705306 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] publish_errors = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.705526 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.705647 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.705772 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.705900 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.706015 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.706128 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.706254 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.706370 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.706484 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.706603 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.706718 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.706831 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] standard_threads = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.706954 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.707069 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.707182 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.707302 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.707416 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.707528 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.707528 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] use_journal = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.707754 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.707873 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] use_json = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.707993 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.708107 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] use_stderr = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.708107 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] use_syslog = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.708334 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.708449 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 28 15:27:08.708562 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.708675 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.708802 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.708917 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.709046 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.709046 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.709254 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.709380 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.709502 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.709616 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.709738 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.709868 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.709985 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.710108 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.710224 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.710343 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.710536 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.710653 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.710772 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.710886 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.710999 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.711121 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.711242 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.711356 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.711477 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.711591 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.711710 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.711834 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.711949 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.712068 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.712188 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.712302 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.712415 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.712541 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.712656 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.712770 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.712770 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.712981 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.713093 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.713215 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.713330 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.713449 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.713564 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.713677 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.713790 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.713929 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.714046 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.714159 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.714279 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.714392 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.714511 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.714635 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.714750 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.714895 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.715519 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.715637 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.715750 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.715872 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.715988 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.716102 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.716221 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.716221 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.716447 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.716633 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.716751 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.716864 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.716991 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.717106 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.717223 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.717345 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.717460 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.717579 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.717700 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.717815 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.717978 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.718103 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.718218 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.718340 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.718461 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.718576 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.718695 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.718805 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.718907 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.719008 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.719112 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.719226 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.719322 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.719427 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.719524 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.719625 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.719751 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.719858 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.719980 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.720094 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.720217 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.720339 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.720454 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.720567 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.720686 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.720810 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.720926 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.721029 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.721141 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.721255 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.721376 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.721491 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.721619 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.721735 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.721871 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.721989 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.722103 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.722216 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.722343 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.722466 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.722581 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.722695 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.722808 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.722922 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.723047 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.723152 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.723265 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.723378 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.723498 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.723613 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.723735 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.723840 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.723953 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.724068 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.724181 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.724307 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.724421 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.724536 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.724651 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.724765 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.724877 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.724997 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.725120 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.725236 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.725356 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.725471 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.725585 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.725705 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.725828 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.725959 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.725959 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.726175 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.726299 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.726415 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.726537 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.726653 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.726766 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.726880 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.726993 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.727114 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.727243 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.727359 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.727473 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.727587 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.727700 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.727815 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.728006 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.728124 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.728239 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.728353 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.728468 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] auth.external = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.728590 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.728722 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.728851 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.728972 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.729088 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.729202 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.729317 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.729431 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] auth.password = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.729555 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.729671 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] auth.token = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.729785 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.729915 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.730031 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.730146 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.730275 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.730389 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.730503 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.730618 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.730732 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.730846 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.731060 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.731214 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.731331 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.731446 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.731550 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.731693 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] credential.caching = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.731818 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.731934 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.732063 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.732186 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.732302 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.732417 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.732531 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.732646 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.732772 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.732895 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.733010 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.733125 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.733240 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.733354 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.733478 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.733595 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.733709 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.733823 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.734035 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.734151 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.734282 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.734400 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.734516 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.734631 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.734744 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.734867 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.734991 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.735107 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.735222 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] auth.external = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.735345 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.735461 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.735577 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] federation.caching = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.735700 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.735818 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.735932 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.736053 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.736168 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] auth.password = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.736283 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.736406 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] auth.token = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.736522 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.736638 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.736754 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.736876 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.736999 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.737124 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.737240 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.737355 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.737470 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.737585 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.737700 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.737824 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.737965 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.738082 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.738196 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.738311 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] credential.caching = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.738425 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity.caching = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.738554 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.738671 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.738787 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.738901 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.738901 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.739124 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.739240 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.739355 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.739469 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.739583 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.739698 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.739821 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.739938 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.740053 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.740167 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.740281 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.740401 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.740527 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.740644 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.740758 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.740873 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.740986 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.741101 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.741226 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.741352 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.741352 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.741567 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.741681 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.741795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.742002 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.742123 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.742239 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.742353 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] federation.caching = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.742353 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.742567 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.742567 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.742792 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.742907 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.743021 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.743135 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.743255 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.743379 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.743495 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.743610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.743724 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.743838 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.743961 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.744085 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.744202 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.744316 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.744430 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.744545 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.744659 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.744782 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.744897 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.745012 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.745126 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.caching = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.745240 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.745354 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.745481 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.745597 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.745712 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.745826 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.745956 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.746073 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.746200 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.746318 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.746433 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.746548 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.746662 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.746776 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.746899 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.747015 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.747131 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.747245 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.747360 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.747475 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.747601 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.747717 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.747831 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.747946 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.748060 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.748176 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.748299 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.748419 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.748535 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.748650 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.748764 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.748879 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.749002 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.749118 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.749234 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.749348 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.749463 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.749463 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.749688 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.749803 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.749936 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.750052 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.750167 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.750295 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.750413 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.750413 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.750628 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.750743 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.750857 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.750980 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.751096 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.751211 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.751326 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.751440 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.751555 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.751555 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.751791 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.751908 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.752022 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.752137 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.752252 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.752375 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.752491 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.752605 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.752719 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.752834 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.752948 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.753070 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.753186 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.753301 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.753416 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.753530 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.753651 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.753774 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.753941 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.754058 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.754173 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.754287 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.754402 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.754530 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.754648 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.754763 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.754877 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.754992 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.755106 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.755229 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.755345 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.755460 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.755854 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.755969 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.756083 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.756206 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.756322 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.756436 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.756551 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.756672 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.756787 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.756911 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.757027 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.757141 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.757141 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.757355 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.757469 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.757592 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.757708 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.757823 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.757953 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.758069 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.758184 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.758311 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.758433 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.758549 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.758665 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.758780 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.758894 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] resource.caching = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.759018 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.759134 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.759250 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.759364 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.759479 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.759594 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.759719 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.759835 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.759950 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.760065 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.760186 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.760303 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.760426 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.760542 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.760657 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.760773 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.760887 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.761002 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.761126 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.761242 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] role.caching = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.761357 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.761472 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] role.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.761587 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.762216 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.762347 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.762465 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.762580 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.762694 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.762810 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.762925 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.763048 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.763164 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.763279 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.763393 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.763508 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.763631 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.763754 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.763871 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.763987 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.764101 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.764216 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.764340 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.764456 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.764571 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.764686 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.764800 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.764916 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.765104 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.765223 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.765338 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.765453 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.765568 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.765683 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.765806 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.765939 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.766056 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.766171 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.766286 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.766401 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.766528 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.766645 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.766761 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.766876 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] resource.caching = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.766990 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.767106 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.767230 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.767346 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.767462 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.767576 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.767691 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.767806 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.767929 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.768052 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.768169 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.768285 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.768403 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.768519 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.768642 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.768759 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.768874 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.768989 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.769103 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.769219 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] role.caching = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.769342 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.769457 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] role.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.769572 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.769686 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.769801 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.769934 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.770062 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] token.caching = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.770181 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.770286 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.770382 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.770477 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.770591 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.770715 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.770831 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.770946 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.771062 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.771176 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.771307 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.771424 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.771545 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.771661 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.771776 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.771890 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.772013 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.772129 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.772251 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.772368 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.772368 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.772588 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.772715 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.772830 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.772945 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.773060 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.773174 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.773288 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.773412 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.773538 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.773666 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.773783 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.773966 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.774084 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.774199 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.connection = **** {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.774313 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.774441 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.774558 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.774674 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.774788 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.774902 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.775016 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.775142 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.775258 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.775373 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.775373 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.775588 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.775711 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.775827 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.775942 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.776056 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.776169 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.776284 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.776410 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.776528 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.776642 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.776757 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.776871 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.776985 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.777108 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.777224 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.777368 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.778005 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.778125 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.778241 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] token.caching = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.778369 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.778493 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.778610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.778726 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.778841 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.778957 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.779081 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81604) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 28 15:27:08.779198 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.779198 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.779198 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.779198 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.779198 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.779198 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.779198 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.779198 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.779198 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.780125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.780125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.780125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.780125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.780125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.780125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.connection = **** {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.780125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.780125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.780975 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.780975 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.780975 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.781299 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.781299 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.781515 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.781706 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.781967 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.782214 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.782452 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.782701 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.782938 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.783184 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.783417 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.783659 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.783897 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 28 15:27:08.784069 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81605) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 28 15:27:08.804914 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:08.805141 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:08.805360 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:08.810345 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:08.810578 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:08.810807 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:08.813701 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:08.819183 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:08.842258 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:08.842484 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:08.846204 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:08.846452 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:08.894893 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:08.896812 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:08.942732 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:08.942963 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:08.944340 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:08.944579 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:09.010160 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:09.010386 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:09.010614 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:09.010838 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:09.011058 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:09.011277 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:09.093306 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:09.093530 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:09.093751 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:09.093992 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:09.094209 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:09.094423 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:09.110859 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:09.111089 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:09.111313 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:09.111531 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:09.111747 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:09.111961 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:09.112175 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:09.123917 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:09.128079 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:09.128342 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:09.128696 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:09.128929 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:09.129155 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:09.129374 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:09.129593 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:09.136845 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:09.137071 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:09.137295 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:09.137517 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:09.137734 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:09.137982 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:09.138065 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:09.138199 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81604) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:09.140859 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:09.141094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:09.141322 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:09.141545 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:09.141763 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:09.142007 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:09.142225 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81605) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 28 15:27:09.155128 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.155290 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 28 15:27:09.155594 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.155676 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.155806 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 28 15:27:09.155886 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 28 15:27:09.156074 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.156155 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.156332 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.156412 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 28 15:27:09.156591 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.156682 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.156848 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.156936 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.157106 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.157367 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.166358 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.166637 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.166907 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.166987 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.167186 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.167268 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.167517 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.167813 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.174768 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:27:09.175533 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:27:09.177625 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:27:09.177931 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.178227 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.178462 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:27:09.178544 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.178750 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.178830 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.179010 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.179091 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.179277 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.179358 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.179548 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.179637 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.179816 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.179895 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.180084 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.180163 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.180348 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.180431 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.180618 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.181006 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.181285 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.202881 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.203900 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.204331 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:27:09.204612 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.205386 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:27:09.205675 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.209220 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:27:09.209500 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.209778 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.210401 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:27:09.210689 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.210976 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.216602 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:27:09.216885 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.217969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:27:09.218259 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.222386 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:27:09.222481 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:27:09.222680 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.222762 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.222969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.223051 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.223257 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.223338 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.223542 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.223670 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.223813 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.223910 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.224083 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.224166 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.224350 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.224432 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.240307 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.240587 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.243260 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.243544 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.244414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:27:09.247451 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:27:09.247726 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.248027 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.250681 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.250981 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.252239 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:27:09.255218 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:27:09.255466 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:27:09.258639 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:27:09.258924 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.259333 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.259512 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.259793 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.260080 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.260376 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.261944 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.262251 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.262541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.262831 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.263124 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.263418 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.279807 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:27:09.280104 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.280368 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.280631 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.284306 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:27:09.284788 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.285219 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.285652 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.288843 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.290402 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.291931 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.292208 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.295514 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:27:09.295797 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.296077 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.296357 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.296758 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.297834 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.300368 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.302908 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.303321 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.307738 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:27:09.307829 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:27:09.308052 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.308146 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.308329 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.308652 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.308736 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.309034 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.309116 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.309333 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.309552 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.309635 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.324907 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:27:09.325874 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:27:09.326290 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.326823 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.327236 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.327650 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.327816 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:27:09.328061 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.328487 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.330712 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.332142 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.333605 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:27:09.333903 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.334300 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.341149 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.342622 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:27:09.345469 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.345867 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.346168 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.346446 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.350323 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:27:09.354167 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:27:09.354449 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.354726 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.354820 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:27:09.358402 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.360717 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.362115 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81604) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:27:09.362397 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.362679 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.362957 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.363236 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.363316 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:27:09.363508 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.363662 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.363785 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.364061 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.364149 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.364345 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.364623 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.364901 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.365160 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.365423 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.365689 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81604) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.372267 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.373754 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:27:09.376790 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.377085 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.377382 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.377658 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.385737 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:27:09.386074 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.386361 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.393890 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81605) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 28 15:27:09.394178 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.394462 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.394741 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.395031 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.395318 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.395596 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.395869 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.396148 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.396431 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.396721 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.396994 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.397263 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.397525 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81605) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 28 15:27:09.400617 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55c7347f49c0 pid: 81604 (default app) Sep 28 15:27:09.403940 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f307fb-155c-4386-9ce2-3675c28a1658 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:09.404221 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f307fb-155c-4386-9ce2-3675c28a1658 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:09.404488 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f307fb-155c-4386-9ce2-3675c28a1658 None None] PATH_INFO: `/v3` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:09.408664 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1/1] 10.222.0.52 () {50 vars in 791 bytes} [Wed Sep 28 15:27:09 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:09.431297 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55c7347f49c0 pid: 81605 (default app) Sep 28 15:27:16.155894 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11147623-d07d-442c-9c61-9dd03a8fe42a None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:16.156418 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11147623-d07d-442c-9c61-9dd03a8fe42a None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:16.156418 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11147623-d07d-442c-9c61-9dd03a8fe42a None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:16.157771 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 2/2] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:27:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:27:16.165155 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b115898-20a6-4a9f-a225-7c6b3a84424c None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:16.165727 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b115898-20a6-4a9f-a225-7c6b3a84424c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:16.165727 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b115898-20a6-4a9f-a225-7c6b3a84424c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:16.236294 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: INFO dbcounter [None req-4b115898-20a6-4a9f-a225-7c6b3a84424c None None] Registered counter for database keystone Sep 28 15:27:16.255512 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG oslo_db.sqlalchemy.engines [None req-4b115898-20a6-4a9f-a225-7c6b3a84424c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81605) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 28 15:27:16.262682 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writer thread running {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 28 15:27:16.437047 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG passlib.handlers.bcrypt [None req-4b115898-20a6-4a9f-a225-7c6b3a84424c None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81605) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 28 15:27:16.437358 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG passlib.handlers.bcrypt [None req-4b115898-20a6-4a9f-a225-7c6b3a84424c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81605) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 28 15:27:16.494844 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-4b115898-20a6-4a9f-a225-7c6b3a84424c None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:27:16.496953 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4b115898-20a6-4a9f-a225-7c6b3a84424c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:27:16.623553 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1/3] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:27:16 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 461 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 28 15:27:17.079029 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11147623-d07d-442c-9c61-9dd03a8fe42a None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:17.149380 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: INFO dbcounter [None req-11147623-d07d-442c-9c61-9dd03a8fe42a None None] Registered counter for database keystone Sep 28 15:27:17.170668 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG oslo_db.sqlalchemy.engines [None req-11147623-d07d-442c-9c61-9dd03a8fe42a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81604) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 28 15:27:17.176695 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writer thread running {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 28 15:27:17.296316 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11147623-d07d-442c-9c61-9dd03a8fe42a None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:17.300592 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff54a09c-6da2-4a26-a8eb-d0e1232d59ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:17.301213 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff54a09c-6da2-4a26-a8eb-d0e1232d59ea admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:17.301490 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff54a09c-6da2-4a26-a8eb-d0e1232d59ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:17.301754 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff54a09c-6da2-4a26-a8eb-d0e1232d59ea admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:17.310477 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff54a09c-6da2-4a26-a8eb-d0e1232d59ea admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:17.362142 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ff54a09c-6da2-4a26-a8eb-d0e1232d59ea admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 28 15:27:17.362425 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff54a09c-6da2-4a26-a8eb-d0e1232d59ea admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:17.371782 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 3/4] 10.222.0.52 () {58 vars in 1324 bytes} [Wed Sep 28 15:27:17 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 28 15:27:17.377835 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b115898-20a6-4a9f-a225-7c6b3a84424c None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:17.392541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b115898-20a6-4a9f-a225-7c6b3a84424c None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:17.396890 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef75f75-40eb-4c6d-b5d2-f2c8429e06c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:17.398005 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef75f75-40eb-4c6d-b5d2-f2c8429e06c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:17.398283 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef75f75-40eb-4c6d-b5d2-f2c8429e06c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:17.398547 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef75f75-40eb-4c6d-b5d2-f2c8429e06c7 admin admin] PATH_INFO: `/v3/projects/8de950cb25d34fe8afed2528d2e112e6` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:17.399969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef75f75-40eb-4c6d-b5d2-f2c8429e06c7 admin admin] RBAC: Authorizing `identity:get_project(project_id=8de950cb25d34fe8afed2528d2e112e6)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:17.452264 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: WARNING keystone.common.rbac_enforcer.enforcer [None req-aef75f75-40eb-4c6d-b5d2-f2c8429e06c7 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 28 15:27:17.452623 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef75f75-40eb-4c6d-b5d2-f2c8429e06c7 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:17.457392 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 2/5] 10.222.0.52 () {56 vars in 1207 bytes} [Wed Sep 28 15:27:17 2022] GET /identity/v3/projects/8de950cb25d34fe8afed2528d2e112e6 => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:19.610345 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a2c943-91d4-481b-947c-068cb4f68eb8 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:19.610883 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a2c943-91d4-481b-947c-068cb4f68eb8 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:19.610883 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a2c943-91d4-481b-947c-068cb4f68eb8 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:19.614935 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 4/6] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:27:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:27:19.620543 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b94f8fa-f7a3-4b8d-8cd6-02e3ff454b3d None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:19.620947 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b94f8fa-f7a3-4b8d-8cd6-02e3ff454b3d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:19.621088 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b94f8fa-f7a3-4b8d-8cd6-02e3ff454b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:19.641093 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-5b94f8fa-f7a3-4b8d-8cd6-02e3ff454b3d None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:27:19.642620 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5b94f8fa-f7a3-4b8d-8cd6-02e3ff454b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:27:19.650360 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 3/7] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:27:19 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 28 15:27:19.915977 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a2c943-91d4-481b-947c-068cb4f68eb8 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:19.927827 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a2c943-91d4-481b-947c-068cb4f68eb8 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:19.931971 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2875bae7-3361-445b-b15b-2bee0f21224c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:19.932557 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875bae7-3361-445b-b15b-2bee0f21224c admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:19.932829 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875bae7-3361-445b-b15b-2bee0f21224c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:19.933094 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875bae7-3361-445b-b15b-2bee0f21224c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:19.940526 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2875bae7-3361-445b-b15b-2bee0f21224c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:19.942324 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2875bae7-3361-445b-b15b-2bee0f21224c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:19.951396 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 5/8] 10.222.0.52 () {58 vars in 1324 bytes} [Wed Sep 28 15:27:19 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 15:27:19.956886 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b94f8fa-f7a3-4b8d-8cd6-02e3ff454b3d None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:19.966372 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b94f8fa-f7a3-4b8d-8cd6-02e3ff454b3d None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:19.970666 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b5ddec-cf94-4dd6-8b66-a61a735e26b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:19.971279 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b5ddec-cf94-4dd6-8b66-a61a735e26b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:19.971554 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b5ddec-cf94-4dd6-8b66-a61a735e26b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:19.971821 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b5ddec-cf94-4dd6-8b66-a61a735e26b5 admin admin] PATH_INFO: `/v3/users/0fdbcf7ba1534603b18de5dbaf96574f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:19.973098 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b5ddec-cf94-4dd6-8b66-a61a735e26b5 admin admin] RBAC: Authorizing `identity:get_user(user_id=0fdbcf7ba1534603b18de5dbaf96574f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:19.974943 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b5ddec-cf94-4dd6-8b66-a61a735e26b5 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:19.976431 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 4/9] 10.222.0.52 () {56 vars in 1198 bytes} [Wed Sep 28 15:27:19 2022] GET /identity/v3/users/0fdbcf7ba1534603b18de5dbaf96574f => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:22.191941 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f6603-7d4e-4fbb-a3c5-5c7151cb6503 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:22.192488 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f6603-7d4e-4fbb-a3c5-5c7151cb6503 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:22.192488 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f6603-7d4e-4fbb-a3c5-5c7151cb6503 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:22.193802 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 6/10] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:27:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:27:22.199829 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef3c024-f50b-4199-9cee-cada6560f806 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:22.200411 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef3c024-f50b-4199-9cee-cada6560f806 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:22.200411 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef3c024-f50b-4199-9cee-cada6560f806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:22.201895 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01944f3-3692-41ed-8292-74c94bd99ff1 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:22.201895 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01944f3-3692-41ed-8292-74c94bd99ff1 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:22.201895 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01944f3-3692-41ed-8292-74c94bd99ff1 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:22.202787 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 7/11] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:27:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:27:22.208430 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2dee3b-3579-41b6-bd57-b87ff92cb0ce None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:22.208710 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2dee3b-3579-41b6-bd57-b87ff92cb0ce None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:22.209034 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2dee3b-3579-41b6-bd57-b87ff92cb0ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:22.219299 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-aef3c024-f50b-4199-9cee-cada6560f806 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:27:22.220494 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-aef3c024-f50b-4199-9cee-cada6560f806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:27:22.259745 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG passlib.handlers.bcrypt [None req-2b2dee3b-3579-41b6-bd57-b87ff92cb0ce None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81604) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 28 15:27:22.260132 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG passlib.handlers.bcrypt [None req-2b2dee3b-3579-41b6-bd57-b87ff92cb0ce None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81604) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 28 15:27:22.300917 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 5/12] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:27:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:27:22.309734 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-2b2dee3b-3579-41b6-bd57-b87ff92cb0ce None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:27:22.311363 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-2b2dee3b-3579-41b6-bd57-b87ff92cb0ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:27:22.418737 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 8/13] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:27:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:27:22.570331 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef3c024-f50b-4199-9cee-cada6560f806 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:22.594466 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:22.673149 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef3c024-f50b-4199-9cee-cada6560f806 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:22.681266 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2dee3b-3579-41b6-bd57-b87ff92cb0ce None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:22.706848 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3faaae-c06e-409f-ae22-a6fe022f7cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:22.707488 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3faaae-c06e-409f-ae22-a6fe022f7cbb None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:22.707775 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3faaae-c06e-409f-ae22-a6fe022f7cbb None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:22.708050 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3faaae-c06e-409f-ae22-a6fe022f7cbb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:22.719954 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3faaae-c06e-409f-ae22-a6fe022f7cbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:27:22.730045 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-4e3faaae-c06e-409f-ae22-a6fe022f7cbb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:27:22.730045 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:27:22.730045 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:27:22.730045 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:27:22.730045 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:27:22.730045 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:27:22.730045 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:27:22.730045 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:27:22.730045 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:27:22.730045 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:27:22.730045 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:27:22.730045 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:27:22.730045 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:27:22.730045 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:27:22.730045 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:27:22.730045 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:27:22.730983 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:27:22.730983 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:27:22.730983 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:27:22.730983 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:27:22.730983 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:27:22.730983 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:27:22.730983 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:27:22.730983 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:27:22.730983 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:27:22.730983 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:27:22.730983 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:27:22.730983 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:27:22.730983 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:27:22.730983 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:27:22.730983 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:27:22.730983 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:27:22.731812 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:27:22.731812 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:27:22.731812 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:27:22.731812 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:27:22.731812 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:27:22.731812 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:27:22.731812 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:27:22.731812 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:27:22.731812 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:27:22.731812 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:27:22.731812 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:27:22.731812 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:27:22.731812 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 6/14] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 15:27:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:27:22.737419 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3faaae-c06e-409f-ae22-a6fe022f7cbb None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:22.784541 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2dee3b-3579-41b6-bd57-b87ff92cb0ce None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:22.786844 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:22.818069 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:22.821162 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6445e6f2-6610-4061-b73f-88ed0d832669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:22.822300 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445e6f2-6610-4061-b73f-88ed0d832669 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:22.822583 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445e6f2-6610-4061-b73f-88ed0d832669 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:22.822850 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445e6f2-6610-4061-b73f-88ed0d832669 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:22.838740 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3faaae-c06e-409f-ae22-a6fe022f7cbb None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:22.872738 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12c521e-d980-4c38-bd21-993aebca573b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:22.873342 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c521e-d980-4c38-bd21-993aebca573b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:22.873623 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c521e-d980-4c38-bd21-993aebca573b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:22.873922 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c521e-d980-4c38-bd21-993aebca573b None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:22.874584 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12c521e-d980-4c38-bd21-993aebca573b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:27:22.874885 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12c521e-d980-4c38-bd21-993aebca573b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:22.876278 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12c521e-d980-4c38-bd21-993aebca573b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:22.888460 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 7/15] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:27:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:27:22.890860 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6445e6f2-6610-4061-b73f-88ed0d832669 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:22.892358 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6445e6f2-6610-4061-b73f-88ed0d832669 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:22.894153 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12c521e-d980-4c38-bd21-993aebca573b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:22.989949 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 9/16] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:27:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 28 15:27:22.990245 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12c521e-d980-4c38-bd21-993aebca573b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:22.995453 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6445e6f2-6610-4061-b73f-88ed0d832669 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:22.998661 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:23.024734 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840958b6-1ca9-4e4e-bf3e-d4278f0c6331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:23.025366 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840958b6-1ca9-4e4e-bf3e-d4278f0c6331 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:23.025648 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840958b6-1ca9-4e4e-bf3e-d4278f0c6331 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:23.025953 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840958b6-1ca9-4e4e-bf3e-d4278f0c6331 None admin] PATH_INFO: `/v3/users/0fdbcf7ba1534603b18de5dbaf96574f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:23.027136 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840958b6-1ca9-4e4e-bf3e-d4278f0c6331 None admin] RBAC: Authorizing `identity:get_user(user_id=0fdbcf7ba1534603b18de5dbaf96574f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:23.028615 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840958b6-1ca9-4e4e-bf3e-d4278f0c6331 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:23.030353 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 8/17] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:27:22 2022] GET /identity/v3/users/0fdbcf7ba1534603b18de5dbaf96574f => generated 247 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:27:23.035665 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840958b6-1ca9-4e4e-bf3e-d4278f0c6331 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:23.050406 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:23.093205 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6445e6f2-6610-4061-b73f-88ed0d832669 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:23.127098 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4629d28-0976-43a0-a28d-851b9fe33221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:23.127973 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4629d28-0976-43a0-a28d-851b9fe33221 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:23.128235 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4629d28-0976-43a0-a28d-851b9fe33221 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:23.128510 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4629d28-0976-43a0-a28d-851b9fe33221 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:23.133765 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840958b6-1ca9-4e4e-bf3e-d4278f0c6331 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:23.147092 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4629d28-0976-43a0-a28d-851b9fe33221 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:23.148549 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4629d28-0976-43a0-a28d-851b9fe33221 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:23.166483 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d75ec-96bc-4261-b84d-8ddc1f69c55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:23.167107 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d75ec-96bc-4261-b84d-8ddc1f69c55b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:23.167307 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-d4629d28-0976-43a0-a28d-851b9fe33221 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:27:23.167307 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:27:23.167307 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:27:23.167307 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:27:23.167307 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:27:23.167307 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:27:23.167307 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:27:23.167307 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:27:23.167307 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:27:23.167307 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:27:23.167307 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:27:23.167307 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:27:23.167307 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:27:23.167307 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:27:23.167307 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:27:23.167307 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:27:23.168709 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:27:23.168709 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:27:23.168709 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:27:23.168709 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:27:23.168709 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:27:23.168709 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:27:23.168709 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:27:23.168709 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:27:23.168709 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:27:23.168709 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:27:23.168709 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:27:23.168709 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:27:23.168709 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:27:23.168709 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:27:23.168709 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:27:23.168709 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:27:23.168709 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:27:23.169503 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:27:23.169503 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:27:23.169503 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:27:23.169661 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d75ec-96bc-4261-b84d-8ddc1f69c55b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:23.169661 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d75ec-96bc-4261-b84d-8ddc1f69c55b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:23.169787 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 10/18] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:27:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:27:23.174310 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5d75ec-96bc-4261-b84d-8ddc1f69c55b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:23.174747 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4629d28-0976-43a0-a28d-851b9fe33221 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:23.175789 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5d75ec-96bc-4261-b84d-8ddc1f69c55b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:23.177346 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 9/19] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:27:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:23.182804 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d75ec-96bc-4261-b84d-8ddc1f69c55b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:23.233217 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:23.262193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:23.277087 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4629d28-0976-43a0-a28d-851b9fe33221 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:23.281811 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d75ec-96bc-4261-b84d-8ddc1f69c55b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:23.315345 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6b2d33-cbc3-4ea7-9e5c-6207b623a5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:23.315519 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1460b65-3144-408c-9c60-58fbbfec65c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:23.316031 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1460b65-3144-408c-9c60-58fbbfec65c9 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:23.316309 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1460b65-3144-408c-9c60-58fbbfec65c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:23.316394 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6b2d33-cbc3-4ea7-9e5c-6207b623a5c4 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:23.316576 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1460b65-3144-408c-9c60-58fbbfec65c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:23.316664 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6b2d33-cbc3-4ea7-9e5c-6207b623a5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:23.316873 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6b2d33-cbc3-4ea7-9e5c-6207b623a5c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:23.317289 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1460b65-3144-408c-9c60-58fbbfec65c9 None admin] RBAC: Adding query filter params (role.id=62d96ac9513c452894fc67b442cfc59f, scope.domain.id=default, user.id=0fdbcf7ba1534603b18de5dbaf96574f) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:27:23.317519 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6b2d33-cbc3-4ea7-9e5c-6207b623a5c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:27:23.317608 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1460b65-3144-408c-9c60-58fbbfec65c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:23.317824 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6b2d33-cbc3-4ea7-9e5c-6207b623a5c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:23.319052 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1460b65-3144-408c-9c60-58fbbfec65c9 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:23.319219 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6b2d33-cbc3-4ea7-9e5c-6207b623a5c4 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:23.325682 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 10/20] 10.222.0.52 () {56 vars in 1322 bytes} [Wed Sep 28 15:27:23 2022] GET /identity/v3/role_assignments?user.id=0fdbcf7ba1534603b18de5dbaf96574f&scope.domain.id=default&role.id=62d96ac9513c452894fc67b442cfc59f => generated 233 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:27:23.337197 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 11/21] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:27:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:27:25.935528 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f843f-feef-49cf-8f0d-8c8e333ce11c None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:25.936466 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f843f-feef-49cf-8f0d-8c8e333ce11c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:25.936918 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f843f-feef-49cf-8f0d-8c8e333ce11c None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:25.938452 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 11/22] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:27:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:27:25.943731 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e25d897-8898-4e0e-af4a-d4b2fdbad51e None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:25.944504 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e25d897-8898-4e0e-af4a-d4b2fdbad51e None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:25.945036 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e25d897-8898-4e0e-af4a-d4b2fdbad51e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:25.974812 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-6e25d897-8898-4e0e-af4a-d4b2fdbad51e None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:27:25.976291 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-6e25d897-8898-4e0e-af4a-d4b2fdbad51e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:27:26.065302 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 12/23] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:27:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:27:26.225469 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c45e46-9524-4726-a5a7-9daabc10a3d4 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:26.226052 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c45e46-9524-4726-a5a7-9daabc10a3d4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:26.226499 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c45e46-9524-4726-a5a7-9daabc10a3d4 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:26.227990 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 12/24] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:27:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:27:26.229609 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422fedf-ccef-47d3-b482-d77cfa71a243 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:26.230147 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422fedf-ccef-47d3-b482-d77cfa71a243 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:26.231165 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422fedf-ccef-47d3-b482-d77cfa71a243 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:26.232645 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 13/25] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:27:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:27:26.237990 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923c873-020a-4019-bfa6-d6251ebcb00d None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:26.238477 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923c873-020a-4019-bfa6-d6251ebcb00d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:26.238924 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923c873-020a-4019-bfa6-d6251ebcb00d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:26.244950 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8843413-1d0a-4707-885c-844ccc36ae91 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:26.247003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8843413-1d0a-4707-885c-844ccc36ae91 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:26.259772 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8843413-1d0a-4707-885c-844ccc36ae91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:26.272377 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-b923c873-020a-4019-bfa6-d6251ebcb00d None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:27:26.274932 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b923c873-020a-4019-bfa6-d6251ebcb00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:27:26.302957 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-b8843413-1d0a-4707-885c-844ccc36ae91 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:27:26.304180 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b8843413-1d0a-4707-885c-844ccc36ae91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:27:26.396531 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:26.431878 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 13/26] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:27:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:27:26.433891 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24119638-4960-49f9-852a-0f676d59ca84 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:26.434177 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24119638-4960-49f9-852a-0f676d59ca84 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:26.434447 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24119638-4960-49f9-852a-0f676d59ca84 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:26.439999 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 14/27] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:27:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:27:26.445982 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 14/28] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:27:26 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:27:26.458301 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8843413-1d0a-4707-885c-844ccc36ae91 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:26.473171 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87c9f11-db57-48e2-83ba-6da4cb82ab7b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:26.473447 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87c9f11-db57-48e2-83ba-6da4cb82ab7b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:26.473710 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87c9f11-db57-48e2-83ba-6da4cb82ab7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:26.499833 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-a87c9f11-db57-48e2-83ba-6da4cb82ab7b None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:27:26.501281 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a87c9f11-db57-48e2-83ba-6da4cb82ab7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:27:26.629353 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8843413-1d0a-4707-885c-844ccc36ae91 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:26.686856 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 15/29] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:27:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:27:26.695471 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc4ab78-1229-439d-8c43-90724d39ea1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:26.696119 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4ab78-1229-439d-8c43-90724d39ea1d None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:26.696398 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4ab78-1229-439d-8c43-90724d39ea1d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:26.696670 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4ab78-1229-439d-8c43-90724d39ea1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:26.712195 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc4ab78-1229-439d-8c43-90724d39ea1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:26.713672 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc4ab78-1229-439d-8c43-90724d39ea1d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:26.736825 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-6fc4ab78-1229-439d-8c43-90724d39ea1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:27:26.736825 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:27:26.736825 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:27:26.736825 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:27:26.736825 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:27:26.736825 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:27:26.736825 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:27:26.736825 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:27:26.736825 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:27:26.736825 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:27:26.736825 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:27:26.736825 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:27:26.736825 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:27:26.736825 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:27:26.736825 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:27:26.736825 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:27:26.737714 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:27:26.737714 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:27:26.737714 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:27:26.737714 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:27:26.737714 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:27:26.737714 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:27:26.737714 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:27:26.737714 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:27:26.737714 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:27:26.737714 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:27:26.737714 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:27:26.737714 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:27:26.737714 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:27:26.737714 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:27:26.737714 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:27:26.737714 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:27:26.737714 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:27:26.738545 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:27:26.738545 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:27:26.738545 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:27:26.743738 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87c9f11-db57-48e2-83ba-6da4cb82ab7b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:26.745996 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 15/30] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:27:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:27:26.751035 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:26.792238 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc4ab78-1229-439d-8c43-90724d39ea1d None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:26.846227 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:26.929362 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87c9f11-db57-48e2-83ba-6da4cb82ab7b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:26.959718 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc4ab78-1229-439d-8c43-90724d39ea1d None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:26.992137 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc36893f-3b42-41c7-9a2a-552346585c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:26.992785 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc36893f-3b42-41c7-9a2a-552346585c2a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:26.993063 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc36893f-3b42-41c7-9a2a-552346585c2a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:26.993330 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc36893f-3b42-41c7-9a2a-552346585c2a None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:26.993987 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc36893f-3b42-41c7-9a2a-552346585c2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:27:26.994282 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc36893f-3b42-41c7-9a2a-552346585c2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:26.995646 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc36893f-3b42-41c7-9a2a-552346585c2a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:27.025838 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76deb2db-1fea-429e-a4d9-ef131f8125d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:27.026386 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76deb2db-1fea-429e-a4d9-ef131f8125d6 None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:27.026657 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76deb2db-1fea-429e-a4d9-ef131f8125d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:27.026922 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76deb2db-1fea-429e-a4d9-ef131f8125d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:27.032632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76deb2db-1fea-429e-a4d9-ef131f8125d6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:27.034098 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76deb2db-1fea-429e-a4d9-ef131f8125d6 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:27.037904 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 16/31] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:27:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:27.038896 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc36893f-3b42-41c7-9a2a-552346585c2a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:27.102191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:27.140700 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc36893f-3b42-41c7-9a2a-552346585c2a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:27.171177 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 16/32] 10.222.0.52 () {60 vars in 1131 bytes} [Wed Sep 28 15:27:26 2022] POST /identity/v3/roles => generated 219 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:27:27.172598 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76deb2db-1fea-429e-a4d9-ef131f8125d6 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:27.177324 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14789f32-41c0-4d69-992d-a63e85758680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:27.178258 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14789f32-41c0-4d69-992d-a63e85758680 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:27.178673 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14789f32-41c0-4d69-992d-a63e85758680 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:27.179079 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14789f32-41c0-4d69-992d-a63e85758680 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:27.194065 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14789f32-41c0-4d69-992d-a63e85758680 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:27:27.199877 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-14789f32-41c0-4d69-992d-a63e85758680 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:27:27.199877 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:27:27.199877 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:27:27.199877 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:27:27.199877 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:27:27.199877 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:27:27.199877 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:27:27.199877 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:27:27.199877 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:27:27.199877 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:27:27.199877 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:27:27.199877 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:27:27.199877 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:27:27.199877 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:27:27.199877 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:27:27.199877 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:27:27.200795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:27:27.200795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:27:27.200795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:27:27.200795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:27:27.200795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:27:27.200795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:27:27.200795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:27:27.200795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:27:27.200795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:27:27.200795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:27:27.200795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:27:27.200795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:27:27.200795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:27:27.200795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:27:27.200795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:27:27.200795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:27:27.201531 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:27:27.201531 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:27:27.201531 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:27:27.201531 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:27:27.201531 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:27:27.201531 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:27:27.201531 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:27:27.201531 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:27:27.201531 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:27:27.201531 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:27:27.201531 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:27:27.201531 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:27:27.209117 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 17/33] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 15:27:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:27:27.211064 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14789f32-41c0-4d69-992d-a63e85758680 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:27.284987 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:27.318445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76deb2db-1fea-429e-a4d9-ef131f8125d6 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:27.331526 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14789f32-41c0-4d69-992d-a63e85758680 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:27.370487 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a15e1-6f81-49b5-aa81-c8289184ffe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:27.374494 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a15e1-6f81-49b5-aa81-c8289184ffe0 None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:27.374797 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a15e1-6f81-49b5-aa81-c8289184ffe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:27.375073 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a15e1-6f81-49b5-aa81-c8289184ffe0 None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:27.376053 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72a15e1-6f81-49b5-aa81-c8289184ffe0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:27.378073 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72a15e1-6f81-49b5-aa81-c8289184ffe0 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:27.385378 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:27.409870 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5127a1c-2ec1-4569-9ab6-2933e2cd6a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:27.410497 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5127a1c-2ec1-4569-9ab6-2933e2cd6a6b None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:27.410782 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5127a1c-2ec1-4569-9ab6-2933e2cd6a6b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:27.411059 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5127a1c-2ec1-4569-9ab6-2933e2cd6a6b None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:27.411742 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5127a1c-2ec1-4569-9ab6-2933e2cd6a6b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:27.413159 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5127a1c-2ec1-4569-9ab6-2933e2cd6a6b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:27.432059 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 17/34] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Sep 28 15:27:27 2022] POST /identity/v3/projects => generated 301 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:27:27.434076 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a15e1-6f81-49b5-aa81-c8289184ffe0 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:27.565939 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a15e1-6f81-49b5-aa81-c8289184ffe0 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:27.566839 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 18/35] 10.222.0.52 () {60 vars in 1131 bytes} [Wed Sep 28 15:27:27 2022] POST /identity/v3/roles => generated 225 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:27:27.568580 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e5797-a8d8-42f0-8469-3ee29f76e92b None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:27.570636 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e5797-a8d8-42f0-8469-3ee29f76e92b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:27.570636 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e5797-a8d8-42f0-8469-3ee29f76e92b None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:27.579822 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 19/36] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:27:27 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:27:27.586155 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a91d68-8189-4ed5-8adc-91ce40045cab None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:27.586598 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a91d68-8189-4ed5-8adc-91ce40045cab None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:27.586975 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a91d68-8189-4ed5-8adc-91ce40045cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:27.621929 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b5ade2-a477-48d2-b4f0-62dfc6e3c3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:27.622837 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b5ade2-a477-48d2-b4f0-62dfc6e3c3ff None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:27.623283 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b5ade2-a477-48d2-b4f0-62dfc6e3c3ff None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:27.623710 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b5ade2-a477-48d2-b4f0-62dfc6e3c3ff None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:27.624555 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b5ade2-a477-48d2-b4f0-62dfc6e3c3ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:27:27.625016 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b5ade2-a477-48d2-b4f0-62dfc6e3c3ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:27.626610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b5ade2-a477-48d2-b4f0-62dfc6e3c3ff None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:27.636082 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-e1a91d68-8189-4ed5-8adc-91ce40045cab None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:27:27.637917 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e1a91d68-8189-4ed5-8adc-91ce40045cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:27:27.651799 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 18/37] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:27:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:27.662907 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b5ade2-a477-48d2-b4f0-62dfc6e3c3ff None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:27.666165 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:27.734510 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 20/38] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:27:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:27:27.736044 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc53d07-52ac-48f5-a57c-15cf05347736 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:27.736656 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc53d07-52ac-48f5-a57c-15cf05347736 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:27.737145 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc53d07-52ac-48f5-a57c-15cf05347736 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:27.738637 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 21/39] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:27:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:27:27.745684 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0002495b-fb0b-43a0-be63-4e64896db489 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:27.746529 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0002495b-fb0b-43a0-be63-4e64896db489 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:27.746892 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0002495b-fb0b-43a0-be63-4e64896db489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:27.772898 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-0002495b-fb0b-43a0-be63-4e64896db489 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:27:27.774305 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0002495b-fb0b-43a0-be63-4e64896db489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:27:27.813981 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b5ade2-a477-48d2-b4f0-62dfc6e3c3ff None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:27.865089 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8261d9-d1e2-4d73-a155-9b148813e4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:27.865673 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8261d9-d1e2-4d73-a155-9b148813e4a4 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:27.865985 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8261d9-d1e2-4d73-a155-9b148813e4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:27.866271 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8261d9-d1e2-4d73-a155-9b148813e4a4 None admin] PATH_INFO: `/v3/users/0fdbcf7ba1534603b18de5dbaf96574f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:27.867649 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8261d9-d1e2-4d73-a155-9b148813e4a4 None admin] RBAC: Authorizing `identity:get_user(user_id=0fdbcf7ba1534603b18de5dbaf96574f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:27.869166 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8261d9-d1e2-4d73-a155-9b148813e4a4 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:27.871653 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 19/40] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:27:27 2022] GET /identity/v3/users/0fdbcf7ba1534603b18de5dbaf96574f => generated 247 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:27:27.876897 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8261d9-d1e2-4d73-a155-9b148813e4a4 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:27.879756 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 22/41] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:27:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:27:27.954694 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:27.988347 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8261d9-d1e2-4d73-a155-9b148813e4a4 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:28.021581 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f2e61d-e9f4-4324-87cd-25b1407ff838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:28.022241 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f2e61d-e9f4-4324-87cd-25b1407ff838 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:28.022519 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f2e61d-e9f4-4324-87cd-25b1407ff838 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:28.022785 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f2e61d-e9f4-4324-87cd-25b1407ff838 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:28.023878 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f2e61d-e9f4-4324-87cd-25b1407ff838 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:28.025311 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f2e61d-e9f4-4324-87cd-25b1407ff838 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:28.027533 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 20/42] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:27:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:27:28.031061 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0002495b-fb0b-43a0-be63-4e64896db489 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:28.033050 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f2e61d-e9f4-4324-87cd-25b1407ff838 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:28.084334 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:28.142539 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f2e61d-e9f4-4324-87cd-25b1407ff838 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:28.146867 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0002495b-fb0b-43a0-be63-4e64896db489 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:28.177351 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be6247e-8cc5-47ea-97ab-61378afe1574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:28.178045 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6247e-8cc5-47ea-97ab-61378afe1574 None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:28.178333 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6247e-8cc5-47ea-97ab-61378afe1574 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:28.178608 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6247e-8cc5-47ea-97ab-61378afe1574 None admin] PATH_INFO: `/v3/domains/default/users/0fdbcf7ba1534603b18de5dbaf96574f/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:28.180523 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be6247e-8cc5-47ea-97ab-61378afe1574 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=0fdbcf7ba1534603b18de5dbaf96574f, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:28.180884 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94937456-5ce7-4178-9f0d-012098f3b5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:28.181576 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94937456-5ce7-4178-9f0d-012098f3b5d7 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:28.181884 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94937456-5ce7-4178-9f0d-012098f3b5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:28.182004 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be6247e-8cc5-47ea-97ab-61378afe1574 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:28.182165 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94937456-5ce7-4178-9f0d-012098f3b5d7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:28.183290 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94937456-5ce7-4178-9f0d-012098f3b5d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:28.184872 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94937456-5ce7-4178-9f0d-012098f3b5d7 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:28.186676 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 23/43] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:27:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:28.187210 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94937456-5ce7-4178-9f0d-012098f3b5d7 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:28.193711 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:28.200000 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 21/44] 10.222.0.52 () {58 vars in 1361 bytes} [Wed Sep 28 15:27:28 2022] PUT /identity/v3/domains/default/users/0fdbcf7ba1534603b18de5dbaf96574f/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:27:28.202714 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be6247e-8cc5-47ea-97ab-61378afe1574 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:28.312942 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94937456-5ce7-4178-9f0d-012098f3b5d7 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:28.330714 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:28.357238 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de04d1c-83ac-4358-9694-5975cdb7e00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:28.357908 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de04d1c-83ac-4358-9694-5975cdb7e00f None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:28.358199 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de04d1c-83ac-4358-9694-5975cdb7e00f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:28.358475 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de04d1c-83ac-4358-9694-5975cdb7e00f None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:28.359169 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de04d1c-83ac-4358-9694-5975cdb7e00f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:28.360697 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de04d1c-83ac-4358-9694-5975cdb7e00f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:28.378047 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 24/45] 10.222.0.52 () {60 vars in 1131 bytes} [Wed Sep 28 15:27:28 2022] POST /identity/v3/roles => generated 223 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:27:28.427062 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be6247e-8cc5-47ea-97ab-61378afe1574 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:28.472095 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78089f9-5328-4235-9bf8-6a63f18ed323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:28.472727 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78089f9-5328-4235-9bf8-6a63f18ed323 None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:28.473009 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78089f9-5328-4235-9bf8-6a63f18ed323 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:28.473284 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78089f9-5328-4235-9bf8-6a63f18ed323 None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:28.473995 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78089f9-5328-4235-9bf8-6a63f18ed323 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:28.475512 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78089f9-5328-4235-9bf8-6a63f18ed323 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:28.506640 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 22/46] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Sep 28 15:27:28 2022] POST /identity/v3/projects => generated 312 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:27:30.368154 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcfa7a6-ba03-4d97-9ba5-73c7ba6bf428 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:30.368676 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcfa7a6-ba03-4d97-9ba5-73c7ba6bf428 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:30.368762 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcfa7a6-ba03-4d97-9ba5-73c7ba6bf428 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:30.370119 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 25/47] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:27:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:27:30.376709 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a16223a-44ab-48f0-99eb-2ccee6c7ed50 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:30.377097 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a16223a-44ab-48f0-99eb-2ccee6c7ed50 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:30.377325 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a16223a-44ab-48f0-99eb-2ccee6c7ed50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:30.400315 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-7a16223a-44ab-48f0-99eb-2ccee6c7ed50 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:27:30.402427 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7a16223a-44ab-48f0-99eb-2ccee6c7ed50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:27:30.449789 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:30.470002 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 23/48] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:27:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:27:30.685282 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6776c-3b54-4618-97a1-7519ea4068bf None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:30.685659 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6776c-3b54-4618-97a1-7519ea4068bf None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:30.686040 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6776c-3b54-4618-97a1-7519ea4068bf None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:30.687457 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 26/49] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:27:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:27:30.692862 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06679c-5984-4d97-9ee1-6a4369014b45 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:30.693142 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06679c-5984-4d97-9ee1-6a4369014b45 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:30.693623 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06679c-5984-4d97-9ee1-6a4369014b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:30.712697 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-ae06679c-5984-4d97-9ee1-6a4369014b45 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:27:30.713898 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ae06679c-5984-4d97-9ee1-6a4369014b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:27:30.741127 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb6776c-3b54-4618-97a1-7519ea4068bf None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:30.780813 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 24/50] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:27:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:27:30.947235 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb6776c-3b54-4618-97a1-7519ea4068bf None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:30.984220 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d476f-3c90-4eb2-a734-b7a0cbd40059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:30.984844 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d476f-3c90-4eb2-a734-b7a0cbd40059 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:30.985123 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d476f-3c90-4eb2-a734-b7a0cbd40059 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:30.985394 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d476f-3c90-4eb2-a734-b7a0cbd40059 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:30.995047 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58d476f-3c90-4eb2-a734-b7a0cbd40059 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:27:30.999888 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-a58d476f-3c90-4eb2-a734-b7a0cbd40059 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:27:30.999888 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:27:30.999888 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:27:30.999888 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:27:30.999888 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:27:30.999888 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:27:30.999888 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:27:30.999888 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:27:30.999888 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:27:30.999888 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:27:30.999888 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:27:30.999888 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:27:30.999888 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:27:30.999888 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:27:30.999888 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:27:30.999888 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:27:31.000706 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:27:31.000706 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:27:31.000706 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:27:31.000706 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:27:31.000706 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:27:31.000706 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:27:31.000706 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:27:31.000706 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:27:31.000706 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:27:31.000706 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:27:31.000706 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:27:31.000706 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:27:31.000706 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:27:31.000706 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:27:31.000706 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:27:31.000706 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:27:31.001472 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:27:31.001472 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:27:31.001472 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:27:31.001472 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:27:31.001472 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:27:31.001472 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:27:31.001472 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:27:31.001472 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:27:31.001472 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:27:31.001472 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:27:31.001472 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:27:31.001472 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:27:31.001472 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 27/51] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 15:27:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:27:31.007009 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06679c-5984-4d97-9ee1-6a4369014b45 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:31.052726 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d476f-3c90-4eb2-a734-b7a0cbd40059 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:31.082739 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:31.104792 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06679c-5984-4d97-9ee1-6a4369014b45 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:31.115880 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:31.140320 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f675013-fa17-4894-b790-c43c9a9127d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:31.140925 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f675013-fa17-4894-b790-c43c9a9127d9 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:31.141202 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f675013-fa17-4894-b790-c43c9a9127d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:31.141473 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f675013-fa17-4894-b790-c43c9a9127d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:31.142157 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f675013-fa17-4894-b790-c43c9a9127d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:27:31.142458 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f675013-fa17-4894-b790-c43c9a9127d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:31.143834 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f675013-fa17-4894-b790-c43c9a9127d9 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:31.151696 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 25/52] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:27:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:31.152934 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d476f-3c90-4eb2-a734-b7a0cbd40059 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:31.157760 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f675013-fa17-4894-b790-c43c9a9127d9 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:31.185389 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13859d21-833c-4768-9a95-3b88239ab0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:31.186039 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13859d21-833c-4768-9a95-3b88239ab0cd None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:31.186316 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13859d21-833c-4768-9a95-3b88239ab0cd None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:31.186583 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13859d21-833c-4768-9a95-3b88239ab0cd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:31.187687 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13859d21-833c-4768-9a95-3b88239ab0cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:31.189131 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13859d21-833c-4768-9a95-3b88239ab0cd None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:31.190686 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 28/53] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:27:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:27:31.196136 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13859d21-833c-4768-9a95-3b88239ab0cd None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:31.257443 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f675013-fa17-4894-b790-c43c9a9127d9 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:31.289146 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:31.293474 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13859d21-833c-4768-9a95-3b88239ab0cd None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:31.293592 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f65179-490d-4cde-8dde-504b62a5aca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:31.293707 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f65179-490d-4cde-8dde-504b62a5aca6 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:31.293987 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f65179-490d-4cde-8dde-504b62a5aca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:31.294264 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f65179-490d-4cde-8dde-504b62a5aca6 None admin] PATH_INFO: `/v3/users/0fdbcf7ba1534603b18de5dbaf96574f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:31.295406 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f65179-490d-4cde-8dde-504b62a5aca6 None admin] RBAC: Authorizing `identity:get_user(user_id=0fdbcf7ba1534603b18de5dbaf96574f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:31.296842 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f65179-490d-4cde-8dde-504b62a5aca6 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:31.298444 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 26/54] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:27:31 2022] GET /identity/v3/users/0fdbcf7ba1534603b18de5dbaf96574f => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:27:31.303716 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f65179-490d-4cde-8dde-504b62a5aca6 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:31.327295 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9563785e-baab-45f5-b0e7-0a2cd81db130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:31.327903 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9563785e-baab-45f5-b0e7-0a2cd81db130 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:31.328189 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9563785e-baab-45f5-b0e7-0a2cd81db130 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:31.328463 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9563785e-baab-45f5-b0e7-0a2cd81db130 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:31.329103 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9563785e-baab-45f5-b0e7-0a2cd81db130 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:31.330578 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9563785e-baab-45f5-b0e7-0a2cd81db130 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:31.344820 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:31.358066 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 29/55] 10.222.0.52 () {60 vars in 1140 bytes} [Wed Sep 28 15:27:31 2022] POST /identity/v3/projects => generated 298 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:27:31.408006 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f65179-490d-4cde-8dde-504b62a5aca6 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:31.444850 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce32203a-e453-4cd1-84f4-ed0bc9f8d6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:31.445503 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce32203a-e453-4cd1-84f4-ed0bc9f8d6c3 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:31.445784 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce32203a-e453-4cd1-84f4-ed0bc9f8d6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:31.446089 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce32203a-e453-4cd1-84f4-ed0bc9f8d6c3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:31.447309 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce32203a-e453-4cd1-84f4-ed0bc9f8d6c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:31.448800 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce32203a-e453-4cd1-84f4-ed0bc9f8d6c3 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:31.450638 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 27/56] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:27:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:27:31.456656 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9563785e-baab-45f5-b0e7-0a2cd81db130 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:31.560819 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9563785e-baab-45f5-b0e7-0a2cd81db130 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:31.593944 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5883bc-62ad-4cba-ad33-265d4709c8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:31.594554 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5883bc-62ad-4cba-ad33-265d4709c8af None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:31.594836 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5883bc-62ad-4cba-ad33-265d4709c8af None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:31.595158 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5883bc-62ad-4cba-ad33-265d4709c8af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:31.595880 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5883bc-62ad-4cba-ad33-265d4709c8af None admin] RBAC: Adding query filter params (role.id=62d96ac9513c452894fc67b442cfc59f, scope.domain.id=default, user.id=0fdbcf7ba1534603b18de5dbaf96574f) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:27:31.596185 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5883bc-62ad-4cba-ad33-265d4709c8af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:31.597629 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5883bc-62ad-4cba-ad33-265d4709c8af None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:31.603314 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 30/57] 10.222.0.52 () {56 vars in 1322 bytes} [Wed Sep 28 15:27:31 2022] GET /identity/v3/role_assignments?user.id=0fdbcf7ba1534603b18de5dbaf96574f&scope.domain.id=default&role.id=62d96ac9513c452894fc67b442cfc59f => generated 530 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:33.488940 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a394c381-14ef-4b55-8e6e-6a2170bffe7c None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:33.489443 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a394c381-14ef-4b55-8e6e-6a2170bffe7c None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:33.489528 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a394c381-14ef-4b55-8e6e-6a2170bffe7c None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:33.490862 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 28/58] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:27:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:27:33.496110 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f7e28-5d14-401c-ab66-a14de2d59348 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:33.496544 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f7e28-5d14-401c-ab66-a14de2d59348 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:33.496657 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f7e28-5d14-401c-ab66-a14de2d59348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:33.515132 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-948f7e28-5d14-401c-ab66-a14de2d59348 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:27:33.516306 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-948f7e28-5d14-401c-ab66-a14de2d59348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:27:33.526521 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:33.580867 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 31/59] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:27:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:27:33.850705 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a394c381-14ef-4b55-8e6e-6a2170bffe7c None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:33.950940 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a394c381-14ef-4b55-8e6e-6a2170bffe7c None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:33.960273 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:33.984566 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1e115-87d5-45aa-af77-03c76dc7e82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:33.985185 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1e115-87d5-45aa-af77-03c76dc7e82f None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:33.985468 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1e115-87d5-45aa-af77-03c76dc7e82f None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:33.985744 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1e115-87d5-45aa-af77-03c76dc7e82f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:33.986887 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a1e115-87d5-45aa-af77-03c76dc7e82f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:33.988315 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a1e115-87d5-45aa-af77-03c76dc7e82f None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:33.989861 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 29/60] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:27:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:33.995332 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948f7e28-5d14-401c-ab66-a14de2d59348 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:34.090867 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948f7e28-5d14-401c-ab66-a14de2d59348 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:34.127412 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aa7790-32bc-486e-b23b-695cc9da91f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:34.127933 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa7790-32bc-486e-b23b-695cc9da91f4 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:34.128216 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa7790-32bc-486e-b23b-695cc9da91f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:34.128491 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa7790-32bc-486e-b23b-695cc9da91f4 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:34.129128 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aa7790-32bc-486e-b23b-695cc9da91f4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:34.130633 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aa7790-32bc-486e-b23b-695cc9da91f4 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:34.157781 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 32/61] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 15:27:33 2022] POST /identity/v3/users => generated 275 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:27:37.454783 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb4dfdf-a34f-4cf4-a449-b9c46f088041 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:37.455808 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb4dfdf-a34f-4cf4-a449-b9c46f088041 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:37.456309 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb4dfdf-a34f-4cf4-a449-b9c46f088041 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:37.457837 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 30/62] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:27:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:27:37.463280 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f57541-a55d-43fb-b137-fc632cf7b0d4 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:37.464067 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f57541-a55d-43fb-b137-fc632cf7b0d4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:37.464566 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f57541-a55d-43fb-b137-fc632cf7b0d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:37.471690 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:37.484588 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-a8f57541-a55d-43fb-b137-fc632cf7b0d4 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:27:37.485808 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a8f57541-a55d-43fb-b137-fc632cf7b0d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:27:37.499154 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0a363-d97a-461c-a9c4-af59b4d72bc9 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:37.499444 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0a363-d97a-461c-a9c4-af59b4d72bc9 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:37.499701 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0a363-d97a-461c-a9c4-af59b4d72bc9 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:37.501218 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 31/63] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:27:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:27:37.519401 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ce177-3ea8-4cb2-86cc-3d1a30e47a54 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:37.519680 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ce177-3ea8-4cb2-86cc-3d1a30e47a54 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:37.519954 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ce177-3ea8-4cb2-86cc-3d1a30e47a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:37.551511 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-cc6ce177-3ea8-4cb2-86cc-3d1a30e47a54 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:27:37.552717 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-cc6ce177-3ea8-4cb2-86cc-3d1a30e47a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:27:37.618112 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 33/64] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:27:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:27:37.680159 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 32/65] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:27:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:27:37.716454 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb782f1-3910-4b7a-86b8-8b44e8df0b55 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:37.716960 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb782f1-3910-4b7a-86b8-8b44e8df0b55 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:37.717377 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb782f1-3910-4b7a-86b8-8b44e8df0b55 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:37.718919 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 34/66] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:27:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:27:37.724420 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf12b185-9e00-4602-8517-907dedceb81e None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:37.724863 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf12b185-9e00-4602-8517-907dedceb81e None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:37.725289 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf12b185-9e00-4602-8517-907dedceb81e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:37.747258 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-bf12b185-9e00-4602-8517-907dedceb81e None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:27:37.748449 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-bf12b185-9e00-4602-8517-907dedceb81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:27:37.841439 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 33/67] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:27:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:27:37.924052 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb782f1-3910-4b7a-86b8-8b44e8df0b55 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:38.065795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb782f1-3910-4b7a-86b8-8b44e8df0b55 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:38.106051 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:38.113241 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f53795-4e84-4dac-9611-76bd5212631e None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:38.113241 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f53795-4e84-4dac-9611-76bd5212631e None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:38.113241 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f53795-4e84-4dac-9611-76bd5212631e None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:38.113241 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 34/68] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:27:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:27:38.115579 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc536ed-a92b-4bee-b86a-63d5dc39d864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:38.116212 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc536ed-a92b-4bee-b86a-63d5dc39d864 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:38.116491 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc536ed-a92b-4bee-b86a-63d5dc39d864 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:38.116764 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc536ed-a92b-4bee-b86a-63d5dc39d864 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:38.121457 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f8dbb-ef70-4302-81b5-c6d7bd477bd8 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:38.122903 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f8dbb-ef70-4302-81b5-c6d7bd477bd8 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:38.123699 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f8dbb-ef70-4302-81b5-c6d7bd477bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:38.138138 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc536ed-a92b-4bee-b86a-63d5dc39d864 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:27:38.143847 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-dcc536ed-a92b-4bee-b86a-63d5dc39d864 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:27:38.143847 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:27:38.143847 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:27:38.143847 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:27:38.143847 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:27:38.143847 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:27:38.143847 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:27:38.143847 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:27:38.143847 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:27:38.143847 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:27:38.143847 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:27:38.143847 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:27:38.143847 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:27:38.143847 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:27:38.143847 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:27:38.143847 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:27:38.144693 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:27:38.144693 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:27:38.144693 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:27:38.144693 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:27:38.144693 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:27:38.144693 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:27:38.144693 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:27:38.144693 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:27:38.144693 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:27:38.144693 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:27:38.144693 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:27:38.144693 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:27:38.144693 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:27:38.144693 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:27:38.144693 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:27:38.144693 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:27:38.145456 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:27:38.145456 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:27:38.145456 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:27:38.145456 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:27:38.145456 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:27:38.145456 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:27:38.145456 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:27:38.145456 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:27:38.145456 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:27:38.145456 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:27:38.145456 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:27:38.145456 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:27:38.147898 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 35/69] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:27:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:27:38.148554 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-411f8dbb-ef70-4302-81b5-c6d7bd477bd8 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:27:38.149152 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc536ed-a92b-4bee-b86a-63d5dc39d864 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:38.150227 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-411f8dbb-ef70-4302-81b5-c6d7bd477bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:27:38.181277 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:38.230623 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 35/70] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:27:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:27:38.239031 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f8dbb-ef70-4302-81b5-c6d7bd477bd8 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:38.259113 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc536ed-a92b-4bee-b86a-63d5dc39d864 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:38.302930 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e6731c-ae9b-4d33-9204-22d054fcd2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:38.303813 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6731c-ae9b-4d33-9204-22d054fcd2ad None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:38.304230 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6731c-ae9b-4d33-9204-22d054fcd2ad None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:38.304639 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6731c-ae9b-4d33-9204-22d054fcd2ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:38.316231 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e6731c-ae9b-4d33-9204-22d054fcd2ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:27:38.321730 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-c6e6731c-ae9b-4d33-9204-22d054fcd2ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:27:38.321730 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:27:38.321730 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:27:38.321730 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:27:38.321730 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:27:38.321730 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:27:38.321730 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:27:38.321730 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:27:38.321730 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:27:38.321730 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:27:38.321730 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:27:38.321730 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:27:38.321730 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:27:38.321730 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:27:38.321730 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:27:38.321730 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:27:38.322575 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:27:38.322575 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:27:38.322575 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:27:38.322575 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:27:38.322575 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:27:38.322575 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:27:38.322575 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:27:38.322575 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:27:38.322575 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:27:38.322575 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:27:38.322575 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:27:38.322575 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:27:38.322575 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:27:38.322575 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:27:38.322575 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:27:38.322575 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:27:38.323367 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:27:38.323367 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:27:38.323367 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:27:38.323367 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:27:38.323367 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:27:38.323367 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:27:38.323367 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:27:38.323367 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:27:38.323367 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:27:38.323367 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:27:38.323367 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:27:38.323367 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:27:38.326723 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 36/71] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 15:27:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:27:38.327740 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e6731c-ae9b-4d33-9204-22d054fcd2ad None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:38.369328 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f8dbb-ef70-4302-81b5-c6d7bd477bd8 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:38.386219 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:38.407618 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916d419c-8010-4710-a936-1b7928076923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:38.408253 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d419c-8010-4710-a936-1b7928076923 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:38.408534 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d419c-8010-4710-a936-1b7928076923 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:38.408804 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d419c-8010-4710-a936-1b7928076923 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:38.421276 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916d419c-8010-4710-a936-1b7928076923 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:27:38.427646 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-916d419c-8010-4710-a936-1b7928076923 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:27:38.427646 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:27:38.427646 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:27:38.427646 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:27:38.427646 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:27:38.427646 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:27:38.427646 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:27:38.427646 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:27:38.427646 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:27:38.427646 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:27:38.427646 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:27:38.427646 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:27:38.427646 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:27:38.427646 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:27:38.427646 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:27:38.427646 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:27:38.428464 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:27:38.428464 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:27:38.428464 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:27:38.428464 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:27:38.428464 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:27:38.428464 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:27:38.428464 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:27:38.428464 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:27:38.428464 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:27:38.428464 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:27:38.428464 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:27:38.428464 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:27:38.428464 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:27:38.428464 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:27:38.428464 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:27:38.428464 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:27:38.429239 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:27:38.429239 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:27:38.429239 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:27:38.429239 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:27:38.429239 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:27:38.429239 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:27:38.429239 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:27:38.429239 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:27:38.429239 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:27:38.429239 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:27:38.429239 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:27:38.429239 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:27:38.429239 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 36/72] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 15:27:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:27:38.429896 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916d419c-8010-4710-a936-1b7928076923 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:38.440815 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e6731c-ae9b-4d33-9204-22d054fcd2ad None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:38.460577 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:38.477969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71c8f4d-0154-4209-8ed7-0c495f60abcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:38.478614 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71c8f4d-0154-4209-8ed7-0c495f60abcc None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:38.479019 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71c8f4d-0154-4209-8ed7-0c495f60abcc None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:38.479293 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71c8f4d-0154-4209-8ed7-0c495f60abcc None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:38.480356 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71c8f4d-0154-4209-8ed7-0c495f60abcc None admin] RBAC: Adding query filter params (name=member) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:27:38.480662 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71c8f4d-0154-4209-8ed7-0c495f60abcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:38.482528 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71c8f4d-0154-4209-8ed7-0c495f60abcc None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:38.492419 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 37/73] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 15:27:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:38.495083 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29d293-106f-4cc4-9c96-d3f494478df7 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:38.495371 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29d293-106f-4cc4-9c96-d3f494478df7 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:38.496245 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29d293-106f-4cc4-9c96-d3f494478df7 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:38.497554 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 38/74] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:27:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:27:38.502148 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b29d293-106f-4cc4-9c96-d3f494478df7 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:38.540576 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916d419c-8010-4710-a936-1b7928076923 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:38.573397 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef78d52-ae5a-47cf-bbb5-1ce9bbd2b1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:38.574206 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef78d52-ae5a-47cf-bbb5-1ce9bbd2b1c0 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:38.574482 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef78d52-ae5a-47cf-bbb5-1ce9bbd2b1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:38.575259 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef78d52-ae5a-47cf-bbb5-1ce9bbd2b1c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:38.575917 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef78d52-ae5a-47cf-bbb5-1ce9bbd2b1c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:27:38.576214 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef78d52-ae5a-47cf-bbb5-1ce9bbd2b1c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:38.577599 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef78d52-ae5a-47cf-bbb5-1ce9bbd2b1c0 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:38.585992 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 37/75] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:27:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:38.586759 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef78d52-ae5a-47cf-bbb5-1ce9bbd2b1c0 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:38.608109 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b29d293-106f-4cc4-9c96-d3f494478df7 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:38.622174 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:38.642933 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e232c4c-8774-45c9-93af-8dadd7200df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:38.643547 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e232c4c-8774-45c9-93af-8dadd7200df6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:38.643813 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e232c4c-8774-45c9-93af-8dadd7200df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:38.644082 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e232c4c-8774-45c9-93af-8dadd7200df6 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:38.644727 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e232c4c-8774-45c9-93af-8dadd7200df6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:27:38.645024 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e232c4c-8774-45c9-93af-8dadd7200df6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:38.646437 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e232c4c-8774-45c9-93af-8dadd7200df6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:38.654443 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 39/76] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:27:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:38.655557 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8061d9-d0b4-4bab-8c32-f37584310927 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:38.655838 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8061d9-d0b4-4bab-8c32-f37584310927 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:38.656106 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8061d9-d0b4-4bab-8c32-f37584310927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:38.673030 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:38.673730 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-9a8061d9-d0b4-4bab-8c32-f37584310927 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:27:38.674939 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9a8061d9-d0b4-4bab-8c32-f37584310927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:27:38.686675 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef78d52-ae5a-47cf-bbb5-1ce9bbd2b1c0 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:38.720755 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539ea63-63b8-49dc-9618-093f5e18a120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:38.721395 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539ea63-63b8-49dc-9618-093f5e18a120 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:38.721677 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539ea63-63b8-49dc-9618-093f5e18a120 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:38.721984 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539ea63-63b8-49dc-9618-093f5e18a120 None admin] PATH_INFO: `/v3/users/b0d4ab9befe041b08d614daa5c9e423c` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:38.737961 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d539ea63-63b8-49dc-9618-093f5e18a120 None admin] RBAC: Authorizing `identity:get_user(user_id=b0d4ab9befe041b08d614daa5c9e423c)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:38.739945 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d539ea63-63b8-49dc-9618-093f5e18a120 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:38.741277 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 40/77] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:27:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:27:38.746704 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 38/78] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:27:38 2022] GET /identity/v3/users/b0d4ab9befe041b08d614daa5c9e423c => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:38.746833 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8061d9-d0b4-4bab-8c32-f37584310927 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:38.746955 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539ea63-63b8-49dc-9618-093f5e18a120 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:38.851047 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8061d9-d0b4-4bab-8c32-f37584310927 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:38.858516 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539ea63-63b8-49dc-9618-093f5e18a120 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:38.861953 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:38.889616 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be4d917-2179-49bc-8ac2-da5ec45520b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:38.890190 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be4d917-2179-49bc-8ac2-da5ec45520b1 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:38.890466 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be4d917-2179-49bc-8ac2-da5ec45520b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:38.890739 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be4d917-2179-49bc-8ac2-da5ec45520b1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:38.897313 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5e95bd-f445-4e9a-9053-ce8f4438a1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:38.898164 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e95bd-f445-4e9a-9053-ce8f4438a1ac None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:38.898626 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e95bd-f445-4e9a-9053-ce8f4438a1ac None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:38.899072 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5e95bd-f445-4e9a-9053-ce8f4438a1ac None admin] PATH_INFO: `/v3/users/0fdbcf7ba1534603b18de5dbaf96574f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:38.900364 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5e95bd-f445-4e9a-9053-ce8f4438a1ac None admin] RBAC: Authorizing `identity:get_user(user_id=0fdbcf7ba1534603b18de5dbaf96574f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:38.901114 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be4d917-2179-49bc-8ac2-da5ec45520b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:27:38.902466 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5e95bd-f445-4e9a-9053-ce8f4438a1ac None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:38.904295 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 39/79] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:27:38 2022] GET /identity/v3/users/0fdbcf7ba1534603b18de5dbaf96574f => generated 247 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:38.905282 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5e95bd-f445-4e9a-9053-ce8f4438a1ac None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:38.906313 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-6be4d917-2179-49bc-8ac2-da5ec45520b1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:27:38.906313 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:27:38.906313 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:27:38.906313 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:27:38.906313 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:27:38.906313 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:27:38.906313 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:27:38.906313 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:27:38.906313 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:27:38.906313 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:27:38.906313 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:27:38.906313 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:27:38.906313 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:27:38.906313 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:27:38.906313 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:27:38.906313 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:27:38.907172 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:27:38.907172 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:27:38.907172 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:27:38.907172 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:27:38.907172 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:27:38.907172 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:27:38.907172 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:27:38.907172 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:27:38.907172 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:27:38.907172 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:27:38.907172 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:27:38.907172 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:27:38.907172 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:27:38.907172 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:27:38.907172 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:27:38.907172 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:27:38.907911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:27:38.907911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:27:38.907911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:27:38.907911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:27:38.907911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:27:38.907911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:27:38.907911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:27:38.907911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:27:38.907911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:27:38.907911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:27:38.907911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:27:38.907911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:27:38.907911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 41/80] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:27:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:27:38.909051 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be4d917-2179-49bc-8ac2-da5ec45520b1 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:38.926874 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:39.014420 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be4d917-2179-49bc-8ac2-da5ec45520b1 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:39.017003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5e95bd-f445-4e9a-9053-ce8f4438a1ac None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:39.047574 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e2fc5-e41d-4a9b-9fdf-c68eeed0340e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:39.048174 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e2fc5-e41d-4a9b-9fdf-c68eeed0340e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:39.048452 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e2fc5-e41d-4a9b-9fdf-c68eeed0340e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:39.048725 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e2fc5-e41d-4a9b-9fdf-c68eeed0340e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:39.050269 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8837d27d-806a-4799-8027-997332201a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:39.050900 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837d27d-806a-4799-8027-997332201a35 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:39.051216 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837d27d-806a-4799-8027-997332201a35 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:39.051513 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837d27d-806a-4799-8027-997332201a35 None admin] PATH_INFO: `/v3/users/b0d4ab9befe041b08d614daa5c9e423c` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:39.052642 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8837d27d-806a-4799-8027-997332201a35 None admin] RBAC: Authorizing `identity:get_user(user_id=b0d4ab9befe041b08d614daa5c9e423c)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:39.054106 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8837d27d-806a-4799-8027-997332201a35 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:39.055667 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 40/81] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:27:38 2022] GET /identity/v3/users/b0d4ab9befe041b08d614daa5c9e423c => generated 275 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:39.056507 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8837d27d-806a-4799-8027-997332201a35 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:39.083767 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:39.119460 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2e2fc5-e41d-4a9b-9fdf-c68eeed0340e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:39.121036 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2e2fc5-e41d-4a9b-9fdf-c68eeed0340e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:39.138206 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:39.157148 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8837d27d-806a-4799-8027-997332201a35 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:39.190180 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8774d0-dba9-4328-ac5f-596454072b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:39.190783 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8774d0-dba9-4328-ac5f-596454072b94 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:39.191059 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8774d0-dba9-4328-ac5f-596454072b94 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:39.191326 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8774d0-dba9-4328-ac5f-596454072b94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:39.220020 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 42/82] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:27:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:27:39.220949 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e2fc5-e41d-4a9b-9fdf-c68eeed0340e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:39.262607 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8774d0-dba9-4328-ac5f-596454072b94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:39.264085 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8774d0-dba9-4328-ac5f-596454072b94 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:39.286906 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:39.321876 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e2fc5-e41d-4a9b-9fdf-c68eeed0340e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:39.344590 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:39.355804 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab81b081-c97e-4ec6-b47d-ead14a303826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:39.356414 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81b081-c97e-4ec6-b47d-ead14a303826 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:39.356699 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81b081-c97e-4ec6-b47d-ead14a303826 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:39.357052 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81b081-c97e-4ec6-b47d-ead14a303826 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:39.357885 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab81b081-c97e-4ec6-b47d-ead14a303826 None admin] RBAC: Adding query filter params (name=member) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:27:39.358426 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab81b081-c97e-4ec6-b47d-ead14a303826 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:39.360728 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab81b081-c97e-4ec6-b47d-ead14a303826 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:39.363985 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 41/83] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:27:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:27:39.364866 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8774d0-dba9-4328-ac5f-596454072b94 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:39.371292 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 43/84] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 15:27:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:39.371910 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab81b081-c97e-4ec6-b47d-ead14a303826 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:39.464471 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8774d0-dba9-4328-ac5f-596454072b94 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:39.469024 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab81b081-c97e-4ec6-b47d-ead14a303826 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:39.495118 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:39.498517 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824b53fb-7924-4312-97b4-8d3a3837d768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:39.499147 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b53fb-7924-4312-97b4-8d3a3837d768 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:39.499432 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b53fb-7924-4312-97b4-8d3a3837d768 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:39.499725 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b53fb-7924-4312-97b4-8d3a3837d768 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:39.500866 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824b53fb-7924-4312-97b4-8d3a3837d768 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:39.502278 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824b53fb-7924-4312-97b4-8d3a3837d768 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:39.503826 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 42/85] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:27:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:39.504000 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a51b5b-402d-4a9b-991f-ca2d4ae4ecf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:39.504572 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824b53fb-7924-4312-97b4-8d3a3837d768 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:39.504679 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a51b5b-402d-4a9b-991f-ca2d4ae4ecf8 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:39.504949 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a51b5b-402d-4a9b-991f-ca2d4ae4ecf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:39.505237 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a51b5b-402d-4a9b-991f-ca2d4ae4ecf8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:39.555920 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:39.572430 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a51b5b-402d-4a9b-991f-ca2d4ae4ecf8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:39.573906 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a51b5b-402d-4a9b-991f-ca2d4ae4ecf8 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:39.605989 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824b53fb-7924-4312-97b4-8d3a3837d768 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:39.639178 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78b431-c1b1-4e23-b928-e2d2652d5822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:39.639809 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78b431-c1b1-4e23-b928-e2d2652d5822 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:39.640091 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78b431-c1b1-4e23-b928-e2d2652d5822 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:39.640362 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78b431-c1b1-4e23-b928-e2d2652d5822 None admin] PATH_INFO: `/v3/projects/4251d157536c4934bf6f66268f3037a9` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:39.641483 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d78b431-c1b1-4e23-b928-e2d2652d5822 None admin] RBAC: Authorizing `identity:get_project(project_id=4251d157536c4934bf6f66268f3037a9)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:39.642949 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d78b431-c1b1-4e23-b928-e2d2652d5822 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:39.644569 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 43/86] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:27:39 2022] GET /identity/v3/projects/4251d157536c4934bf6f66268f3037a9 => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:39.645314 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78b431-c1b1-4e23-b928-e2d2652d5822 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:39.675317 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 44/87] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:27:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:27:39.676168 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a51b5b-402d-4a9b-991f-ca2d4ae4ecf8 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:39.703819 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:39.747692 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78b431-c1b1-4e23-b928-e2d2652d5822 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:39.763835 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:39.775394 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a51b5b-402d-4a9b-991f-ca2d4ae4ecf8 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:39.781553 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2677779-67d5-42c6-9d8a-a2e25495162b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:39.782218 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2677779-67d5-42c6-9d8a-a2e25495162b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:39.782499 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2677779-67d5-42c6-9d8a-a2e25495162b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:39.782773 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2677779-67d5-42c6-9d8a-a2e25495162b None admin] PATH_INFO: `/v3/projects/4251d157536c4934bf6f66268f3037a9` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:39.783894 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2677779-67d5-42c6-9d8a-a2e25495162b None admin] RBAC: Authorizing `identity:get_project(project_id=4251d157536c4934bf6f66268f3037a9)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:39.785342 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2677779-67d5-42c6-9d8a-a2e25495162b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:39.787167 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 44/88] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:27:39 2022] GET /identity/v3/projects/4251d157536c4934bf6f66268f3037a9 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:39.787713 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2677779-67d5-42c6-9d8a-a2e25495162b None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:39.815289 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd2a46e-990c-4146-8100-569a2be32e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:39.815912 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2a46e-990c-4146-8100-569a2be32e2a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:39.816192 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2a46e-990c-4146-8100-569a2be32e2a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:39.816461 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2a46e-990c-4146-8100-569a2be32e2a None admin] PATH_INFO: `/v3/users/b0d4ab9befe041b08d614daa5c9e423c` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:39.817568 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd2a46e-990c-4146-8100-569a2be32e2a None admin] RBAC: Authorizing `identity:get_user(user_id=b0d4ab9befe041b08d614daa5c9e423c)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:39.819045 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd2a46e-990c-4146-8100-569a2be32e2a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:39.820751 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 45/89] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:27:39 2022] GET /identity/v3/users/b0d4ab9befe041b08d614daa5c9e423c => generated 275 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:39.821499 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd2a46e-990c-4146-8100-569a2be32e2a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:39.885720 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2677779-67d5-42c6-9d8a-a2e25495162b None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:39.915330 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:39.922320 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865299ba-ce87-4eaf-b144-4d738a50fcd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:39.922466 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd2a46e-990c-4146-8100-569a2be32e2a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:39.922933 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865299ba-ce87-4eaf-b144-4d738a50fcd5 None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:39.923218 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865299ba-ce87-4eaf-b144-4d738a50fcd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:39.923496 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865299ba-ce87-4eaf-b144-4d738a50fcd5 None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:39.924149 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865299ba-ce87-4eaf-b144-4d738a50fcd5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:39.925651 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865299ba-ce87-4eaf-b144-4d738a50fcd5 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:39.952135 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 45/90] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 15:27:39 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:27:39.953046 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865299ba-ce87-4eaf-b144-4d738a50fcd5 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:39.956128 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a8ec79-e9bc-4b09-b5a1-e057965dfcd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:39.956859 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8ec79-e9bc-4b09-b5a1-e057965dfcd2 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:39.957168 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8ec79-e9bc-4b09-b5a1-e057965dfcd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:39.957577 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8ec79-e9bc-4b09-b5a1-e057965dfcd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:39.958493 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a8ec79-e9bc-4b09-b5a1-e057965dfcd2 None admin] RBAC: Adding query filter params (role.id=e152a36b5a1b470b912b1d705462f512, scope.project.id=4251d157536c4934bf6f66268f3037a9, user.id=b0d4ab9befe041b08d614daa5c9e423c) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:27:39.958866 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a8ec79-e9bc-4b09-b5a1-e057965dfcd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:39.960449 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a8ec79-e9bc-4b09-b5a1-e057965dfcd2 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:39.974897 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 46/91] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 15:27:39 2022] GET /identity/v3/role_assignments?user.id=b0d4ab9befe041b08d614daa5c9e423c&scope.project.id=4251d157536c4934bf6f66268f3037a9&role.id=e152a36b5a1b470b912b1d705462f512 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:39.975910 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a8ec79-e9bc-4b09-b5a1-e057965dfcd2 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:40.024441 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:40.100802 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865299ba-ce87-4eaf-b144-4d738a50fcd5 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:40.126890 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a8ec79-e9bc-4b09-b5a1-e057965dfcd2 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:40.146203 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eea4530-8182-4ffa-928a-efedb6c6751f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:40.147292 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea4530-8182-4ffa-928a-efedb6c6751f None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:40.148033 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea4530-8182-4ffa-928a-efedb6c6751f None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:40.148740 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea4530-8182-4ffa-928a-efedb6c6751f None admin] PATH_INFO: `/v3/projects/4251d157536c4934bf6f66268f3037a9` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:40.150298 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eea4530-8182-4ffa-928a-efedb6c6751f None admin] RBAC: Authorizing `identity:get_project(project_id=4251d157536c4934bf6f66268f3037a9)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:40.153433 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eea4530-8182-4ffa-928a-efedb6c6751f None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:40.157036 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 46/92] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:27:39 2022] GET /identity/v3/projects/4251d157536c4934bf6f66268f3037a9 => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:40.158163 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eea4530-8182-4ffa-928a-efedb6c6751f None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:40.169242 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd411768-7f48-4357-bde5-3523b0ea930c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:40.169879 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd411768-7f48-4357-bde5-3523b0ea930c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:40.170206 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd411768-7f48-4357-bde5-3523b0ea930c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:40.171014 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd411768-7f48-4357-bde5-3523b0ea930c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:40.171260 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd411768-7f48-4357-bde5-3523b0ea930c None admin] RBAC: Adding query filter params (role.id=62d96ac9513c452894fc67b442cfc59f, scope.project.id=4251d157536c4934bf6f66268f3037a9, user.id=0fdbcf7ba1534603b18de5dbaf96574f) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:27:40.171581 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd411768-7f48-4357-bde5-3523b0ea930c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:40.174434 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd411768-7f48-4357-bde5-3523b0ea930c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:40.181239 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 47/93] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 15:27:39 2022] GET /identity/v3/role_assignments?user.id=0fdbcf7ba1534603b18de5dbaf96574f&scope.project.id=4251d157536c4934bf6f66268f3037a9&role.id=62d96ac9513c452894fc67b442cfc59f => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:40.181895 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd411768-7f48-4357-bde5-3523b0ea930c None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:40.216108 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:40.296877 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eea4530-8182-4ffa-928a-efedb6c6751f None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:40.305322 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd411768-7f48-4357-bde5-3523b0ea930c None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:40.310294 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:40.338566 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7240963-22f2-47ad-b411-f7119621d070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:40.339198 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7240963-22f2-47ad-b411-f7119621d070 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:40.340040 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7240963-22f2-47ad-b411-f7119621d070 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:40.340314 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7240963-22f2-47ad-b411-f7119621d070 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:40.345267 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e089d986-50d6-490d-a0f0-6691f07afd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:40.345924 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e089d986-50d6-490d-a0f0-6691f07afd3d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:40.346212 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e089d986-50d6-490d-a0f0-6691f07afd3d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:40.346489 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e089d986-50d6-490d-a0f0-6691f07afd3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:40.347233 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e089d986-50d6-490d-a0f0-6691f07afd3d None admin] RBAC: Adding query filter params (role.id=5038c8140b7145ceb7494210a5c9f9f0, scope.project.id=4251d157536c4934bf6f66268f3037a9, user.id=b0d4ab9befe041b08d614daa5c9e423c) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:27:40.347540 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e089d986-50d6-490d-a0f0-6691f07afd3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:40.349059 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e089d986-50d6-490d-a0f0-6691f07afd3d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:40.355992 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 48/94] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 15:27:40 2022] GET /identity/v3/role_assignments?user.id=b0d4ab9befe041b08d614daa5c9e423c&scope.project.id=4251d157536c4934bf6f66268f3037a9&role.id=5038c8140b7145ceb7494210a5c9f9f0 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:40.457025 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7240963-22f2-47ad-b411-f7119621d070 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:40.458603 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7240963-22f2-47ad-b411-f7119621d070 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:40.582319 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 47/95] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:27:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:27:40.587982 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e089d986-50d6-490d-a0f0-6691f07afd3d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:40.685020 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:40.697335 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e089d986-50d6-490d-a0f0-6691f07afd3d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:40.733800 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ffcd48-66c6-40e6-a400-7912c0f87dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:40.734465 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffcd48-66c6-40e6-a400-7912c0f87dd2 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:40.734751 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffcd48-66c6-40e6-a400-7912c0f87dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:40.735029 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffcd48-66c6-40e6-a400-7912c0f87dd2 None admin] PATH_INFO: `/v3/projects/17a69146ef934856b8533fdb8ddfd873` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:40.736168 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ffcd48-66c6-40e6-a400-7912c0f87dd2 None admin] RBAC: Authorizing `identity:get_project(project_id=17a69146ef934856b8533fdb8ddfd873)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:40.737620 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ffcd48-66c6-40e6-a400-7912c0f87dd2 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:40.739462 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 49/96] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:27:40 2022] GET /identity/v3/projects/17a69146ef934856b8533fdb8ddfd873 => generated 312 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:40.745749 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7240963-22f2-47ad-b411-f7119621d070 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:40.758032 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:40.870107 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7240963-22f2-47ad-b411-f7119621d070 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:40.913018 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72993928-9cde-462b-863e-224df51521a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:40.913693 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72993928-9cde-462b-863e-224df51521a0 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:40.914000 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72993928-9cde-462b-863e-224df51521a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:40.914276 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72993928-9cde-462b-863e-224df51521a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:40.914998 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72993928-9cde-462b-863e-224df51521a0 None admin] RBAC: Adding query filter params (role.id=e152a36b5a1b470b912b1d705462f512, scope.project.id=17a69146ef934856b8533fdb8ddfd873, user.id=b0d4ab9befe041b08d614daa5c9e423c) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:27:40.915306 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72993928-9cde-462b-863e-224df51521a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:40.916751 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72993928-9cde-462b-863e-224df51521a0 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:40.933698 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 48/97] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 15:27:40 2022] GET /identity/v3/role_assignments?user.id=b0d4ab9befe041b08d614daa5c9e423c&scope.project.id=17a69146ef934856b8533fdb8ddfd873&role.id=e152a36b5a1b470b912b1d705462f512 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:43.236345 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e458ffb-aa2a-41a3-a99c-0aa8dbfc1fb4 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:43.237776 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e458ffb-aa2a-41a3-a99c-0aa8dbfc1fb4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:43.238616 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e458ffb-aa2a-41a3-a99c-0aa8dbfc1fb4 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:43.241662 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 50/98] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:27:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:27:43.247800 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794eda37-0a54-4b2a-8608-c7dab6f9c78d None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:43.249060 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794eda37-0a54-4b2a-8608-c7dab6f9c78d None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:43.249996 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794eda37-0a54-4b2a-8608-c7dab6f9c78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:43.311219 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-794eda37-0a54-4b2a-8608-c7dab6f9c78d None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:27:43.313407 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-794eda37-0a54-4b2a-8608-c7dab6f9c78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:27:43.326802 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09e366-b696-4cf9-aa6c-ae80b6c7a9d3 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:43.327649 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09e366-b696-4cf9-aa6c-ae80b6c7a9d3 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:43.328380 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09e366-b696-4cf9-aa6c-ae80b6c7a9d3 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:43.330731 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 51/99] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:27:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:27:43.338452 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc999115-5bff-40a4-94bd-79bede66f567 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:43.339822 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc999115-5bff-40a4-94bd-79bede66f567 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:43.340580 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc999115-5bff-40a4-94bd-79bede66f567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:43.392419 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-fc999115-5bff-40a4-94bd-79bede66f567 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:27:43.400227 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-fc999115-5bff-40a4-94bd-79bede66f567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:27:43.544148 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 49/100] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:27:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:27:43.563674 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 52/101] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:27:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:27:43.859703 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794eda37-0a54-4b2a-8608-c7dab6f9c78d None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:43.863336 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:43.881782 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b351c9ab-9a4b-43da-8e8f-3de6fcc65bf3 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:43.882102 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b351c9ab-9a4b-43da-8e8f-3de6fcc65bf3 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:43.882535 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b351c9ab-9a4b-43da-8e8f-3de6fcc65bf3 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:43.884013 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 53/102] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:27:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:27:43.893332 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c3175-17be-4e50-a528-391dc1c049d3 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:43.893332 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c3175-17be-4e50-a528-391dc1c049d3 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:43.893332 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c3175-17be-4e50-a528-391dc1c049d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:43.913561 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-8b0c3175-17be-4e50-a528-391dc1c049d3 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:27:43.916109 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8b0c3175-17be-4e50-a528-391dc1c049d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:27:44.059382 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 54/103] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:27:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:27:44.061776 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794eda37-0a54-4b2a-8608-c7dab6f9c78d None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:44.101582 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020549d2-98a9-4cc8-b1be-c8ebb3b3e355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:44.102624 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020549d2-98a9-4cc8-b1be-c8ebb3b3e355 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:44.103102 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020549d2-98a9-4cc8-b1be-c8ebb3b3e355 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:44.103595 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020549d2-98a9-4cc8-b1be-c8ebb3b3e355 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:44.116181 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020549d2-98a9-4cc8-b1be-c8ebb3b3e355 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:27:44.121453 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-020549d2-98a9-4cc8-b1be-c8ebb3b3e355 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:27:44.121453 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:27:44.121453 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:27:44.121453 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:27:44.121453 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:27:44.121453 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:27:44.121453 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:27:44.121453 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:27:44.121453 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:27:44.121453 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:27:44.121453 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:27:44.121453 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:27:44.121453 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:27:44.121453 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:27:44.121453 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:27:44.121453 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:27:44.122299 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:27:44.122299 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:27:44.122299 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:27:44.122299 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:27:44.122299 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:27:44.122299 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:27:44.122299 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:27:44.122299 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:27:44.122299 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:27:44.122299 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:27:44.122299 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:27:44.122299 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:27:44.122299 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:27:44.122299 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:27:44.122299 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:27:44.122299 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:27:44.123077 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:27:44.123077 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:27:44.123077 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:27:44.123077 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:27:44.123077 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:27:44.123077 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:27:44.123077 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:27:44.123077 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:27:44.123077 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:27:44.123077 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:27:44.123077 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:27:44.123077 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:27:44.126314 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c3175-17be-4e50-a528-391dc1c049d3 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:44.127075 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 50/104] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:27:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:27:44.132299 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020549d2-98a9-4cc8-b1be-c8ebb3b3e355 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:44.161885 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:44.209731 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:44.260321 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020549d2-98a9-4cc8-b1be-c8ebb3b3e355 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:44.302654 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b39889-6f5c-41ce-9409-9baa22376dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:44.303915 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b39889-6f5c-41ce-9409-9baa22376dbc None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:44.304393 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b39889-6f5c-41ce-9409-9baa22376dbc None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:44.304875 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b39889-6f5c-41ce-9409-9baa22376dbc None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:44.305730 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b39889-6f5c-41ce-9409-9baa22376dbc None admin] RBAC: Adding query filter params (name=member) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:27:44.306907 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b39889-6f5c-41ce-9409-9baa22376dbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:44.309316 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b39889-6f5c-41ce-9409-9baa22376dbc None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:44.319586 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c3175-17be-4e50-a528-391dc1c049d3 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:44.321073 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 51/105] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 15:27:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:27:44.323572 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f56fcf7-fc1e-4a53-8081-706f8d695065 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:44.324494 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f56fcf7-fc1e-4a53-8081-706f8d695065 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:44.324867 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f56fcf7-fc1e-4a53-8081-706f8d695065 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:44.326476 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 52/106] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:27:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:27:44.327614 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2983f12-1faf-4c05-ada7-0328b6cac1a5 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:44.327991 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2983f12-1faf-4c05-ada7-0328b6cac1a5 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:44.328358 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2983f12-1faf-4c05-ada7-0328b6cac1a5 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:44.329887 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 53/107] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:27:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:27:44.330776 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2983f12-1faf-4c05-ada7-0328b6cac1a5 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:44.364203 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4264d8ce-1526-4c01-ab51-8de75db7c00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:44.365147 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4264d8ce-1526-4c01-ab51-8de75db7c00a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:44.365572 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4264d8ce-1526-4c01-ab51-8de75db7c00a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:44.366036 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4264d8ce-1526-4c01-ab51-8de75db7c00a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:44.379171 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4264d8ce-1526-4c01-ab51-8de75db7c00a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:27:44.384769 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-4264d8ce-1526-4c01-ab51-8de75db7c00a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:27:44.384769 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:27:44.384769 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:27:44.384769 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:27:44.384769 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:27:44.384769 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:27:44.384769 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:27:44.384769 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:27:44.384769 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:27:44.384769 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:27:44.384769 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:27:44.384769 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:27:44.384769 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:27:44.384769 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:27:44.384769 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:27:44.384769 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:27:44.385690 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:27:44.385690 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:27:44.385690 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:27:44.385690 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:27:44.385690 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:27:44.385690 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:27:44.385690 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:27:44.385690 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:27:44.385690 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:27:44.385690 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:27:44.385690 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:27:44.385690 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:27:44.385690 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:27:44.385690 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:27:44.385690 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:27:44.385690 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:27:44.387064 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:27:44.387064 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:27:44.387064 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:27:44.387064 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:27:44.387064 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:27:44.387064 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:27:44.387064 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:27:44.387064 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:27:44.387064 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:27:44.387064 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:27:44.387064 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:27:44.387064 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:27:44.389415 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 55/108] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 15:27:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:27:44.391033 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e407c6e-f2cc-41eb-85dd-66b76346b036 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:44.391532 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e407c6e-f2cc-41eb-85dd-66b76346b036 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:44.391998 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e407c6e-f2cc-41eb-85dd-66b76346b036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:44.410929 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-7e407c6e-f2cc-41eb-85dd-66b76346b036 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:27:44.412429 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7e407c6e-f2cc-41eb-85dd-66b76346b036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:27:44.463863 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2983f12-1faf-4c05-ada7-0328b6cac1a5 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:44.481232 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 56/109] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:27:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:27:44.486026 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a746ff0c-7911-4e6c-a4e5-e2dfc7b9bf90 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:44.486026 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a746ff0c-7911-4e6c-a4e5-e2dfc7b9bf90 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:44.486026 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a746ff0c-7911-4e6c-a4e5-e2dfc7b9bf90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:44.490252 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:44.499589 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62320db-b7d0-44bf-a1fc-5411734c92c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:44.500217 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62320db-b7d0-44bf-a1fc-5411734c92c5 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:44.500500 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62320db-b7d0-44bf-a1fc-5411734c92c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:44.500768 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62320db-b7d0-44bf-a1fc-5411734c92c5 None admin] PATH_INFO: `/v3/users/b0d4ab9befe041b08d614daa5c9e423c` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:44.501914 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62320db-b7d0-44bf-a1fc-5411734c92c5 None admin] RBAC: Authorizing `identity:get_user(user_id=b0d4ab9befe041b08d614daa5c9e423c)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:44.503016 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:44.503352 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62320db-b7d0-44bf-a1fc-5411734c92c5 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:44.505254 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 54/110] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:27:44 2022] GET /identity/v3/users/b0d4ab9befe041b08d614daa5c9e423c => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:44.506544 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62320db-b7d0-44bf-a1fc-5411734c92c5 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:44.510848 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-a746ff0c-7911-4e6c-a4e5-e2dfc7b9bf90 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:27:44.512601 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a746ff0c-7911-4e6c-a4e5-e2dfc7b9bf90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:27:44.581819 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 57/111] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:27:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:27:44.590902 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a746ff0c-7911-4e6c-a4e5-e2dfc7b9bf90 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:44.626605 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62320db-b7d0-44bf-a1fc-5411734c92c5 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:44.667858 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a32265-3f1b-4378-9ecc-e61c25bab35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:44.668757 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a32265-3f1b-4378-9ecc-e61c25bab35a None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:44.669227 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a32265-3f1b-4378-9ecc-e61c25bab35a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:44.669640 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a32265-3f1b-4378-9ecc-e61c25bab35a None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:44.670564 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a32265-3f1b-4378-9ecc-e61c25bab35a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:27:44.671008 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a32265-3f1b-4378-9ecc-e61c25bab35a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:44.672725 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a32265-3f1b-4378-9ecc-e61c25bab35a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:44.681681 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 55/112] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:27:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:44.683212 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fade40-7e21-40a2-96b3-ba021deeeca6 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:44.683643 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fade40-7e21-40a2-96b3-ba021deeeca6 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:44.684055 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fade40-7e21-40a2-96b3-ba021deeeca6 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:44.685427 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 56/113] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:27:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:27:44.690465 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fade40-7e21-40a2-96b3-ba021deeeca6 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:44.708223 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a746ff0c-7911-4e6c-a4e5-e2dfc7b9bf90 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:44.744267 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea02e23e-45fb-4b79-a352-a1706c2878b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:44.745159 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea02e23e-45fb-4b79-a352-a1706c2878b8 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:44.745633 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea02e23e-45fb-4b79-a352-a1706c2878b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:44.746075 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea02e23e-45fb-4b79-a352-a1706c2878b8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:44.750594 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:44.754033 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:44.759737 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea02e23e-45fb-4b79-a352-a1706c2878b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:27:44.765108 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-ea02e23e-45fb-4b79-a352-a1706c2878b8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:27:44.765108 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:27:44.765108 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:27:44.765108 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:27:44.765108 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:27:44.765108 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:27:44.765108 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:27:44.765108 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:27:44.765108 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:27:44.765108 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:27:44.765108 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:27:44.765108 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:27:44.765108 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:27:44.765108 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:27:44.765108 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:27:44.765108 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:27:44.766674 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:27:44.766674 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:27:44.766674 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:27:44.766674 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:27:44.766674 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:27:44.766674 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:27:44.766674 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:27:44.766674 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:27:44.766674 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:27:44.766674 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:27:44.766674 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:27:44.766674 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:27:44.766674 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:27:44.766674 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:27:44.766674 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:27:44.766674 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:27:44.767451 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:27:44.767451 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:27:44.767451 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:27:44.767451 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:27:44.767451 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:27:44.767451 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:27:44.767451 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:27:44.767451 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:27:44.767451 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:27:44.767451 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:27:44.767451 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:27:44.767451 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:27:44.767451 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 58/114] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 15:27:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:27:44.767451 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea02e23e-45fb-4b79-a352-a1706c2878b8 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:44.809386 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fade40-7e21-40a2-96b3-ba021deeeca6 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:44.845825 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f455b8e-62e8-486d-8562-018a6b35a3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:44.846527 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f455b8e-62e8-486d-8562-018a6b35a3cd None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:44.846801 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f455b8e-62e8-486d-8562-018a6b35a3cd None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:44.847121 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f455b8e-62e8-486d-8562-018a6b35a3cd None admin] PATH_INFO: `/v3/projects/4251d157536c4934bf6f66268f3037a9` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:44.848254 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f455b8e-62e8-486d-8562-018a6b35a3cd None admin] RBAC: Authorizing `identity:get_project(project_id=4251d157536c4934bf6f66268f3037a9)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:44.849636 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f455b8e-62e8-486d-8562-018a6b35a3cd None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:44.851530 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 57/115] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:27:44 2022] GET /identity/v3/projects/4251d157536c4934bf6f66268f3037a9 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:44.852612 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c44380-0585-41d6-8af7-eb1a58b9d47d None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:44.852897 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c44380-0585-41d6-8af7-eb1a58b9d47d None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:44.853217 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c44380-0585-41d6-8af7-eb1a58b9d47d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:44.870528 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-f3c44380-0585-41d6-8af7-eb1a58b9d47d None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:27:44.871728 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-f3c44380-0585-41d6-8af7-eb1a58b9d47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:27:44.876338 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea02e23e-45fb-4b79-a352-a1706c2878b8 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:44.910593 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6189f8a6-50ab-4a00-9812-a1def9f50239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:44.911222 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189f8a6-50ab-4a00-9812-a1def9f50239 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:44.911505 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189f8a6-50ab-4a00-9812-a1def9f50239 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:44.911780 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189f8a6-50ab-4a00-9812-a1def9f50239 None admin] PATH_INFO: `/v3/users/0fdbcf7ba1534603b18de5dbaf96574f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:44.912901 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6189f8a6-50ab-4a00-9812-a1def9f50239 None admin] RBAC: Authorizing `identity:get_user(user_id=0fdbcf7ba1534603b18de5dbaf96574f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:44.914382 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6189f8a6-50ab-4a00-9812-a1def9f50239 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:44.915949 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 59/116] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:27:44 2022] GET /identity/v3/users/0fdbcf7ba1534603b18de5dbaf96574f => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:44.920818 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6189f8a6-50ab-4a00-9812-a1def9f50239 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:44.935606 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 58/117] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:27:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:27:44.936420 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c44380-0585-41d6-8af7-eb1a58b9d47d None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:44.981714 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:45.013154 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:45.024010 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6189f8a6-50ab-4a00-9812-a1def9f50239 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:45.062613 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b41a4b0-dd9b-4b8f-8b4f-f99424c98fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:45.063243 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b41a4b0-dd9b-4b8f-8b4f-f99424c98fe2 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:45.063522 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b41a4b0-dd9b-4b8f-8b4f-f99424c98fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:45.063790 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b41a4b0-dd9b-4b8f-8b4f-f99424c98fe2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:45.064101 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c44380-0585-41d6-8af7-eb1a58b9d47d None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:45.064901 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b41a4b0-dd9b-4b8f-8b4f-f99424c98fe2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:45.066385 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b41a4b0-dd9b-4b8f-8b4f-f99424c98fe2 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:45.067932 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 60/118] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:27:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:45.068685 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b41a4b0-dd9b-4b8f-8b4f-f99424c98fe2 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:45.100993 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d0656d-26f5-4f7d-9423-a337a39ee888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:45.101602 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d0656d-26f5-4f7d-9423-a337a39ee888 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:45.101904 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d0656d-26f5-4f7d-9423-a337a39ee888 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:45.102184 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d0656d-26f5-4f7d-9423-a337a39ee888 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:45.102839 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d0656d-26f5-4f7d-9423-a337a39ee888 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:27:45.103138 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d0656d-26f5-4f7d-9423-a337a39ee888 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:45.104503 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d0656d-26f5-4f7d-9423-a337a39ee888 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:45.113379 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 59/119] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:27:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:45.114160 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d0656d-26f5-4f7d-9423-a337a39ee888 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:45.175085 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b41a4b0-dd9b-4b8f-8b4f-f99424c98fe2 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:45.205899 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:45.211957 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288ae57-7798-4d50-ba95-5a3498962d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:45.212590 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288ae57-7798-4d50-ba95-5a3498962d61 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:45.212872 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288ae57-7798-4d50-ba95-5a3498962d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:45.213146 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288ae57-7798-4d50-ba95-5a3498962d61 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:45.221087 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d0656d-26f5-4f7d-9423-a337a39ee888 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:45.223997 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b288ae57-7798-4d50-ba95-5a3498962d61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:27:45.228770 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-b288ae57-7798-4d50-ba95-5a3498962d61 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 15:27:45.228770 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:27:45.228770 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:27:45.228770 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:27:45.228770 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:27:45.228770 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:27:45.228770 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:27:45.228770 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:27:45.228770 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:27:45.228770 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:27:45.228770 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:27:45.228770 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:27:45.228770 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:27:45.228770 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:27:45.228770 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:27:45.228770 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:27:45.229580 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:27:45.229580 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:27:45.229580 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:27:45.229580 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:27:45.229580 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:27:45.229580 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:27:45.229580 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:27:45.229580 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:27:45.229580 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:27:45.229580 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:27:45.229580 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:27:45.229580 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:27:45.229580 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:27:45.229580 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:27:45.229580 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:27:45.229580 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:27:45.231049 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:27:45.231049 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:27:45.231049 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:27:45.231049 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:27:45.231049 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:27:45.231049 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:27:45.231049 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:27:45.231049 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:27:45.231049 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:27:45.231049 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:27:45.231049 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 15:27:45.231049 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:27:45.231049 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 61/120] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:27:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:27:45.232234 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288ae57-7798-4d50-ba95-5a3498962d61 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:45.239055 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:45.257342 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be76455-fa1b-4eb6-b720-2007cf633668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:45.258027 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be76455-fa1b-4eb6-b720-2007cf633668 None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:45.258309 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be76455-fa1b-4eb6-b720-2007cf633668 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:45.258578 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be76455-fa1b-4eb6-b720-2007cf633668 None admin] PATH_INFO: `/v3/projects/4251d157536c4934bf6f66268f3037a9/users/b0d4ab9befe041b08d614daa5c9e423c/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:45.260806 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be76455-fa1b-4eb6-b720-2007cf633668 None admin] RBAC: Authorizing `identity:create_grant(project_id=4251d157536c4934bf6f66268f3037a9, user_id=b0d4ab9befe041b08d614daa5c9e423c, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:45.262325 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be76455-fa1b-4eb6-b720-2007cf633668 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:45.279528 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 60/121] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 15:27:45 2022] PUT /identity/v3/projects/4251d157536c4934bf6f66268f3037a9/users/b0d4ab9befe041b08d614daa5c9e423c/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:27:45.280345 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be76455-fa1b-4eb6-b720-2007cf633668 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:45.340389 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288ae57-7798-4d50-ba95-5a3498962d61 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:45.381495 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4969f32f-9417-4ef3-9a83-32c3c79ad371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:45.382175 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969f32f-9417-4ef3-9a83-32c3c79ad371 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:45.382455 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969f32f-9417-4ef3-9a83-32c3c79ad371 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:45.383198 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969f32f-9417-4ef3-9a83-32c3c79ad371 None admin] PATH_INFO: `/v3/projects/4251d157536c4934bf6f66268f3037a9` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:45.384804 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4969f32f-9417-4ef3-9a83-32c3c79ad371 None admin] RBAC: Authorizing `identity:get_project(project_id=4251d157536c4934bf6f66268f3037a9)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:45.387245 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4969f32f-9417-4ef3-9a83-32c3c79ad371 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:45.389612 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 62/122] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:27:45 2022] GET /identity/v3/projects/4251d157536c4934bf6f66268f3037a9 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:45.390462 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4969f32f-9417-4ef3-9a83-32c3c79ad371 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:45.413537 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be76455-fa1b-4eb6-b720-2007cf633668 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:45.461301 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17edfa2a-6e0e-4157-a667-d183c445b261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:45.461948 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17edfa2a-6e0e-4157-a667-d183c445b261 None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:45.462236 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17edfa2a-6e0e-4157-a667-d183c445b261 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:45.462429 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:45.462525 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17edfa2a-6e0e-4157-a667-d183c445b261 None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:45.463692 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17edfa2a-6e0e-4157-a667-d183c445b261 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:45.466045 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17edfa2a-6e0e-4157-a667-d183c445b261 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:45.502262 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 61/123] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Sep 28 15:27:45 2022] POST /identity/v3/projects => generated 302 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:27:45.502262 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17edfa2a-6e0e-4157-a667-d183c445b261 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:45.503687 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4969f32f-9417-4ef3-9a83-32c3c79ad371 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:45.549231 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:45.552878 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f10320-007a-4e13-b5a8-ab73aab6ca85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:45.553591 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f10320-007a-4e13-b5a8-ab73aab6ca85 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:45.553903 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f10320-007a-4e13-b5a8-ab73aab6ca85 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:45.554148 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f10320-007a-4e13-b5a8-ab73aab6ca85 None admin] PATH_INFO: `/v3/users/b0d4ab9befe041b08d614daa5c9e423c` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:45.555381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f10320-007a-4e13-b5a8-ab73aab6ca85 None admin] RBAC: Authorizing `identity:get_user(user_id=b0d4ab9befe041b08d614daa5c9e423c)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:45.556885 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f10320-007a-4e13-b5a8-ab73aab6ca85 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:45.559005 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 63/124] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:27:45 2022] GET /identity/v3/users/b0d4ab9befe041b08d614daa5c9e423c => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:45.559644 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f10320-007a-4e13-b5a8-ab73aab6ca85 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:45.646820 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17edfa2a-6e0e-4157-a667-d183c445b261 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:45.696378 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f10320-007a-4e13-b5a8-ab73aab6ca85 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:45.696598 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd254c92-7766-4890-959c-79540ac60fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:45.697088 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd254c92-7766-4890-959c-79540ac60fa5 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:45.697365 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd254c92-7766-4890-959c-79540ac60fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:45.697635 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd254c92-7766-4890-959c-79540ac60fa5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:45.712126 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd254c92-7766-4890-959c-79540ac60fa5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:27:45.717962 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-bd254c92-7766-4890-959c-79540ac60fa5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:27:45.717962 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:27:45.717962 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:27:45.717962 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:27:45.717962 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:27:45.717962 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:27:45.717962 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:27:45.717962 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:27:45.717962 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:27:45.717962 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:27:45.717962 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:27:45.717962 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:27:45.717962 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:27:45.717962 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:27:45.717962 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:27:45.717962 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:27:45.719019 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:27:45.719019 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:27:45.719019 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:27:45.719019 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:27:45.719019 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:27:45.719019 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:27:45.719019 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:27:45.719019 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:27:45.719019 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:27:45.719019 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:27:45.719019 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:27:45.719019 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:27:45.719019 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:27:45.719019 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:27:45.719019 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:27:45.719019 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:27:45.719848 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:27:45.719848 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:27:45.719848 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:27:45.719848 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:27:45.719848 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:27:45.719848 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:27:45.719848 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:27:45.719848 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:27:45.719848 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:27:45.719848 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:27:45.719848 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:27:45.719848 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:27:45.719848 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 62/125] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:27:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:27:45.720473 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd254c92-7766-4890-959c-79540ac60fa5 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:45.731089 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:45.744663 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f71b138-5e6d-4f58-9451-73260fe33b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:45.745280 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71b138-5e6d-4f58-9451-73260fe33b01 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:45.746103 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71b138-5e6d-4f58-9451-73260fe33b01 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:45.746377 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71b138-5e6d-4f58-9451-73260fe33b01 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:45.747064 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f71b138-5e6d-4f58-9451-73260fe33b01 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:27:45.747369 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f71b138-5e6d-4f58-9451-73260fe33b01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:45.748751 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f71b138-5e6d-4f58-9451-73260fe33b01 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:45.759056 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 64/126] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 15:27:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:45.762081 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f71b138-5e6d-4f58-9451-73260fe33b01 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:45.840837 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:45.855654 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd254c92-7766-4890-959c-79540ac60fa5 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:45.890176 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f71b138-5e6d-4f58-9451-73260fe33b01 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:45.902789 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94a67b5-d7a9-446e-9405-365394795ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:45.904185 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a67b5-d7a9-446e-9405-365394795ffc None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:45.904703 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a67b5-d7a9-446e-9405-365394795ffc None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:45.905112 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a67b5-d7a9-446e-9405-365394795ffc None admin] PATH_INFO: `/v3/projects/4251d157536c4934bf6f66268f3037a9/users/0fdbcf7ba1534603b18de5dbaf96574f/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:45.908179 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94a67b5-d7a9-446e-9405-365394795ffc None admin] RBAC: Authorizing `identity:create_grant(project_id=4251d157536c4934bf6f66268f3037a9, user_id=0fdbcf7ba1534603b18de5dbaf96574f, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:45.910399 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94a67b5-d7a9-446e-9405-365394795ffc None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:45.934243 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13422d3f-27ac-4dfc-9117-b82b43a8ea7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:45.935026 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13422d3f-27ac-4dfc-9117-b82b43a8ea7e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:45.935411 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13422d3f-27ac-4dfc-9117-b82b43a8ea7e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:45.935787 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13422d3f-27ac-4dfc-9117-b82b43a8ea7e None admin] PATH_INFO: `/v3/projects/4251d157536c4934bf6f66268f3037a9` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:45.937023 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13422d3f-27ac-4dfc-9117-b82b43a8ea7e None admin] RBAC: Authorizing `identity:get_project(project_id=4251d157536c4934bf6f66268f3037a9)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:45.937687 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 63/127] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 15:27:45 2022] PUT /identity/v3/projects/4251d157536c4934bf6f66268f3037a9/users/0fdbcf7ba1534603b18de5dbaf96574f/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:27:45.941925 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13422d3f-27ac-4dfc-9117-b82b43a8ea7e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:45.941925 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 65/128] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:27:45 2022] GET /identity/v3/projects/4251d157536c4934bf6f66268f3037a9 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:45.941925 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13422d3f-27ac-4dfc-9117-b82b43a8ea7e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:45.943295 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94a67b5-d7a9-446e-9405-365394795ffc None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:46.064864 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:46.114939 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13422d3f-27ac-4dfc-9117-b82b43a8ea7e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:46.156236 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b2fa97-82f0-419e-9b7d-46e423a23095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:46.156858 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b2fa97-82f0-419e-9b7d-46e423a23095 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:46.157138 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b2fa97-82f0-419e-9b7d-46e423a23095 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:46.157411 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b2fa97-82f0-419e-9b7d-46e423a23095 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:46.158103 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b2fa97-82f0-419e-9b7d-46e423a23095 None admin] RBAC: Adding query filter params (name=member) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:27:46.158403 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b2fa97-82f0-419e-9b7d-46e423a23095 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:46.159865 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b2fa97-82f0-419e-9b7d-46e423a23095 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:46.168396 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94a67b5-d7a9-446e-9405-365394795ffc None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:46.173876 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 66/129] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 15:27:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:46.174721 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b2fa97-82f0-419e-9b7d-46e423a23095 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:46.221070 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68405cad-9088-4421-baf7-fe048cfbb258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:46.222229 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68405cad-9088-4421-baf7-fe048cfbb258 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:46.222731 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68405cad-9088-4421-baf7-fe048cfbb258 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:46.223232 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68405cad-9088-4421-baf7-fe048cfbb258 None admin] PATH_INFO: `/v3/users/9f3930cf37644c92bae454a9ddcc1670` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:46.240043 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68405cad-9088-4421-baf7-fe048cfbb258 None admin] RBAC: Authorizing `identity:get_user(user_id=9f3930cf37644c92bae454a9ddcc1670)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:46.242530 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68405cad-9088-4421-baf7-fe048cfbb258 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:46.245275 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 64/130] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:27:45 2022] GET /identity/v3/users/9f3930cf37644c92bae454a9ddcc1670 => generated 289 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:46.246595 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68405cad-9088-4421-baf7-fe048cfbb258 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:46.259017 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:46.310558 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b2fa97-82f0-419e-9b7d-46e423a23095 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:46.331261 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:46.345964 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe7f897-5396-4a1f-ae45-9d2036572bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:46.346621 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe7f897-5396-4a1f-ae45-9d2036572bdd None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:46.346911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe7f897-5396-4a1f-ae45-9d2036572bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:46.347189 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe7f897-5396-4a1f-ae45-9d2036572bdd None admin] PATH_INFO: `/v3/projects/4251d157536c4934bf6f66268f3037a9/users/b0d4ab9befe041b08d614daa5c9e423c/roles/5038c8140b7145ceb7494210a5c9f9f0` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:46.349377 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe7f897-5396-4a1f-ae45-9d2036572bdd None admin] RBAC: Authorizing `identity:create_grant(project_id=4251d157536c4934bf6f66268f3037a9, user_id=b0d4ab9befe041b08d614daa5c9e423c, role_id=5038c8140b7145ceb7494210a5c9f9f0)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:46.350863 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe7f897-5396-4a1f-ae45-9d2036572bdd None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:46.365178 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68405cad-9088-4421-baf7-fe048cfbb258 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:46.374590 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 67/131] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 15:27:46 2022] PUT /identity/v3/projects/4251d157536c4934bf6f66268f3037a9/users/b0d4ab9befe041b08d614daa5c9e423c/roles/5038c8140b7145ceb7494210a5c9f9f0 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:27:46.375204 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe7f897-5396-4a1f-ae45-9d2036572bdd None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:46.470167 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94123190-6d9f-45a0-bed6-e27ccde451ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:46.470821 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94123190-6d9f-45a0-bed6-e27ccde451ad None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:46.471102 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94123190-6d9f-45a0-bed6-e27ccde451ad None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:46.471377 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94123190-6d9f-45a0-bed6-e27ccde451ad None admin] PATH_INFO: `/v3/users/b0d4ab9befe041b08d614daa5c9e423c` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:46.472520 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94123190-6d9f-45a0-bed6-e27ccde451ad None admin] RBAC: Authorizing `identity:get_user(user_id=b0d4ab9befe041b08d614daa5c9e423c)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:46.474062 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94123190-6d9f-45a0-bed6-e27ccde451ad None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:46.475951 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 65/132] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:27:46 2022] GET /identity/v3/users/b0d4ab9befe041b08d614daa5c9e423c => generated 275 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:46.482194 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94123190-6d9f-45a0-bed6-e27ccde451ad None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:46.571836 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe7f897-5396-4a1f-ae45-9d2036572bdd None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:46.609811 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5969a56c-6faf-4ccc-97cd-6e121cc85a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:46.610381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969a56c-6faf-4ccc-97cd-6e121cc85a2d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:46.610670 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969a56c-6faf-4ccc-97cd-6e121cc85a2d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:46.610947 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969a56c-6faf-4ccc-97cd-6e121cc85a2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:46.662894 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:46.665763 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:46.682966 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5969a56c-6faf-4ccc-97cd-6e121cc85a2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:46.684476 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5969a56c-6faf-4ccc-97cd-6e121cc85a2d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:46.739094 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94123190-6d9f-45a0-bed6-e27ccde451ad None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:46.801861 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae1f135-98a5-4824-ad8b-700fd8af952d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:46.802927 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1f135-98a5-4824-ad8b-700fd8af952d None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:46.803445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1f135-98a5-4824-ad8b-700fd8af952d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:46.803906 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae1f135-98a5-4824-ad8b-700fd8af952d None admin] PATH_INFO: `/v3/projects/17a69146ef934856b8533fdb8ddfd873` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:46.805223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae1f135-98a5-4824-ad8b-700fd8af952d None admin] RBAC: Authorizing `identity:get_project(project_id=17a69146ef934856b8533fdb8ddfd873)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:46.807120 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae1f135-98a5-4824-ad8b-700fd8af952d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:46.809555 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 66/133] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:27:46 2022] GET /identity/v3/projects/17a69146ef934856b8533fdb8ddfd873 => generated 312 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:27:46.815500 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae1f135-98a5-4824-ad8b-700fd8af952d None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:46.816659 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 68/134] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:27:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:27:46.822124 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5969a56c-6faf-4ccc-97cd-6e121cc85a2d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:46.911141 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:46.924408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5969a56c-6faf-4ccc-97cd-6e121cc85a2d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:46.964003 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c114468-1a5d-4a92-be07-2f37936a3820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:46.964780 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c114468-1a5d-4a92-be07-2f37936a3820 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:46.965321 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c114468-1a5d-4a92-be07-2f37936a3820 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:46.965805 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c114468-1a5d-4a92-be07-2f37936a3820 None admin] PATH_INFO: `/v3/projects/4251d157536c4934bf6f66268f3037a9` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:46.967511 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c114468-1a5d-4a92-be07-2f37936a3820 None admin] RBAC: Authorizing `identity:get_project(project_id=4251d157536c4934bf6f66268f3037a9)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:46.968992 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c114468-1a5d-4a92-be07-2f37936a3820 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:46.970676 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 69/135] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:27:46 2022] GET /identity/v3/projects/4251d157536c4934bf6f66268f3037a9 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:27:46.982443 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c114468-1a5d-4a92-be07-2f37936a3820 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:47.000004 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae1f135-98a5-4824-ad8b-700fd8af952d None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:47.083954 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d94844-dea9-4457-a6a5-21558d9eca7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:47.084624 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d94844-dea9-4457-a6a5-21558d9eca7b None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:47.084925 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d94844-dea9-4457-a6a5-21558d9eca7b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:47.085178 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d94844-dea9-4457-a6a5-21558d9eca7b None admin] PATH_INFO: `/v3/projects/17a69146ef934856b8533fdb8ddfd873/users/b0d4ab9befe041b08d614daa5c9e423c/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:47.089052 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d94844-dea9-4457-a6a5-21558d9eca7b None admin] RBAC: Authorizing `identity:create_grant(project_id=17a69146ef934856b8533fdb8ddfd873, user_id=b0d4ab9befe041b08d614daa5c9e423c, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:47.091102 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d94844-dea9-4457-a6a5-21558d9eca7b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:47.118363 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 67/136] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 15:27:46 2022] PUT /identity/v3/projects/17a69146ef934856b8533fdb8ddfd873/users/b0d4ab9befe041b08d614daa5c9e423c/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:27:47.158360 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c114468-1a5d-4a92-be07-2f37936a3820 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:47.195608 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1597a855-4900-43cc-b615-1c6e865bc2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:47.196157 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1597a855-4900-43cc-b615-1c6e865bc2f1 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:47.196505 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1597a855-4900-43cc-b615-1c6e865bc2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:47.196812 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1597a855-4900-43cc-b615-1c6e865bc2f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:47.197558 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1597a855-4900-43cc-b615-1c6e865bc2f1 None admin] RBAC: Adding query filter params (role.id=9e4cabcdb1e349c699da538ce00d1396, scope.project.id=4251d157536c4934bf6f66268f3037a9, user.id=9f3930cf37644c92bae454a9ddcc1670) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:27:47.197892 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1597a855-4900-43cc-b615-1c6e865bc2f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:47.200109 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1597a855-4900-43cc-b615-1c6e865bc2f1 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:47.206668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 70/137] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 15:27:46 2022] GET /identity/v3/role_assignments?user.id=9f3930cf37644c92bae454a9ddcc1670&scope.project.id=4251d157536c4934bf6f66268f3037a9&role.id=9e4cabcdb1e349c699da538ce00d1396 => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:48.174716 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54738e85-9419-4acf-bbe4-60f134d87091 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:48.175238 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54738e85-9419-4acf-bbe4-60f134d87091 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:48.175344 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54738e85-9419-4acf-bbe4-60f134d87091 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:48.176919 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 68/138] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:27:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:27:48.181993 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3f589-2b52-4ae9-b59b-6c308eee6f0d None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:48.182408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3f589-2b52-4ae9-b59b-6c308eee6f0d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:48.182594 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c3f589-2b52-4ae9-b59b-6c308eee6f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:48.206979 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-a5c3f589-2b52-4ae9-b59b-6c308eee6f0d None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:27:48.208202 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a5c3f589-2b52-4ae9-b59b-6c308eee6f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:27:48.216948 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:48.303772 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 71/139] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:27:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:27:48.408550 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6b38a-cb75-4a8f-adce-4d1e5acfb868 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:48.408848 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6b38a-cb75-4a8f-adce-4d1e5acfb868 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:48.409094 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6b38a-cb75-4a8f-adce-4d1e5acfb868 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:48.410738 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 69/140] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:27:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:27:48.416260 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41987a7-7358-4324-8800-fc4f3e2273f0 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:48.416547 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41987a7-7358-4324-8800-fc4f3e2273f0 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:48.416833 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41987a7-7358-4324-8800-fc4f3e2273f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:48.442905 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d41987a7-7358-4324-8800-fc4f3e2273f0 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:27:48.444075 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d41987a7-7358-4324-8800-fc4f3e2273f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:27:48.527419 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 72/141] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:27:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:27:48.694280 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb71c62-d506-4deb-813e-6d2e67c6a11a None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:48.694641 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb71c62-d506-4deb-813e-6d2e67c6a11a None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:48.694814 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb71c62-d506-4deb-813e-6d2e67c6a11a None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:48.696295 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 70/142] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:27:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:27:48.702913 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4b65f-637c-4d50-9e1a-6eece6b94554 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:48.703252 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4b65f-637c-4d50-9e1a-6eece6b94554 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:48.703538 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4b65f-637c-4d50-9e1a-6eece6b94554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:48.733356 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-59f4b65f-637c-4d50-9e1a-6eece6b94554 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:27:48.734556 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-59f4b65f-637c-4d50-9e1a-6eece6b94554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:27:48.821999 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 73/143] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:27:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:27:48.887845 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb71c62-d506-4deb-813e-6d2e67c6a11a None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:48.890815 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:48.907949 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f4b65f-637c-4d50-9e1a-6eece6b94554 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:48.910590 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:49.111395 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb71c62-d506-4deb-813e-6d2e67c6a11a None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:49.126731 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f4b65f-637c-4d50-9e1a-6eece6b94554 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:49.197314 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5260f159-e8b0-42f7-906c-791bb38a00c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:49.197993 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260f159-e8b0-42f7-906c-791bb38a00c9 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:49.198252 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260f159-e8b0-42f7-906c-791bb38a00c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:49.198566 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5260f159-e8b0-42f7-906c-791bb38a00c9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:49.200778 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5260f159-e8b0-42f7-906c-791bb38a00c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:49.210559 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5260f159-e8b0-42f7-906c-791bb38a00c9 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:49.214657 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1a72e8-5b1e-4f86-b7f3-597d14c9bb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:49.215739 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a72e8-5b1e-4f86-b7f3-597d14c9bb45 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:49.216231 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a72e8-5b1e-4f86-b7f3-597d14c9bb45 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:49.216727 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a72e8-5b1e-4f86-b7f3-597d14c9bb45 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:49.217226 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 71/144] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:27:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:49.219629 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5260f159-e8b0-42f7-906c-791bb38a00c9 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:49.253880 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1a72e8-5b1e-4f86-b7f3-597d14c9bb45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:27:49.259174 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-8b1a72e8-5b1e-4f86-b7f3-597d14c9bb45 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:27:49.259174 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:27:49.259174 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:27:49.259174 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:27:49.259174 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:27:49.259174 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:27:49.259174 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:27:49.259174 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:27:49.259174 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:27:49.259174 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:27:49.259174 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:27:49.259174 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:27:49.259174 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:27:49.259174 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:27:49.259174 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:27:49.259174 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:27:49.262195 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:27:49.262195 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:27:49.262195 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:27:49.262195 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:27:49.262195 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:27:49.262195 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:27:49.262195 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:27:49.262195 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:27:49.262195 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:27:49.262195 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:27:49.262195 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:27:49.262195 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:27:49.262195 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:27:49.262195 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:27:49.262195 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:27:49.262195 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:27:49.264082 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:27:49.264082 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:27:49.264082 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:27:49.264082 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:27:49.264082 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:27:49.264082 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:27:49.264082 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:27:49.264082 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:27:49.264082 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:27:49.264082 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:27:49.264082 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:27:49.264082 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:27:49.264082 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 74/145] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:27:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:27:49.264082 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1a72e8-5b1e-4f86-b7f3-597d14c9bb45 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:49.355973 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:49.390388 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:49.417964 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1a72e8-5b1e-4f86-b7f3-597d14c9bb45 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:49.424778 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5260f159-e8b0-42f7-906c-791bb38a00c9 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:49.487111 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433512b3-a286-4956-8b31-628922539fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:49.488407 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433512b3-a286-4956-8b31-628922539fbf None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:49.489369 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433512b3-a286-4956-8b31-628922539fbf None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:49.490197 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf05991-ddf0-4640-8a57-e78e4556098a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:49.491063 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf05991-ddf0-4640-8a57-e78e4556098a None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:49.491576 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433512b3-a286-4956-8b31-628922539fbf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:49.492500 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf05991-ddf0-4640-8a57-e78e4556098a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:49.493130 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf05991-ddf0-4640-8a57-e78e4556098a None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:49.498655 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf05991-ddf0-4640-8a57-e78e4556098a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:49.504639 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf05991-ddf0-4640-8a57-e78e4556098a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:49.513678 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433512b3-a286-4956-8b31-628922539fbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:27:49.518552 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-433512b3-a286-4956-8b31-628922539fbf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:27:49.518552 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:27:49.518552 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:27:49.518552 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:27:49.518552 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:27:49.518552 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:27:49.518552 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:27:49.518552 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:27:49.518552 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:27:49.518552 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:27:49.518552 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:27:49.518552 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:27:49.518552 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:27:49.518552 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:27:49.518552 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:27:49.518552 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:27:49.519407 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:27:49.519407 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:27:49.519407 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:27:49.519407 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:27:49.519407 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:27:49.519407 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:27:49.519407 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:27:49.519407 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:27:49.519407 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:27:49.519407 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:27:49.519407 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:27:49.519407 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:27:49.519407 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:27:49.519407 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:27:49.519407 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:27:49.519407 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:27:49.520142 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:27:49.520142 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:27:49.520142 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:27:49.520142 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:27:49.520142 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:27:49.520142 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:27:49.520142 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:27:49.520142 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:27:49.520142 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:27:49.520142 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:27:49.520142 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:27:49.520142 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:27:49.520142 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 75/146] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 15:27:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:27:49.520769 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433512b3-a286-4956-8b31-628922539fbf None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:49.557083 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 72/147] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 15:27:49 2022] POST /identity/v3/users => generated 283 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:27:49.557887 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf05991-ddf0-4640-8a57-e78e4556098a None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:49.631998 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433512b3-a286-4956-8b31-628922539fbf None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:49.670899 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c674b4dc-71cd-4196-b5a4-fe4513047741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:49.671899 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c674b4dc-71cd-4196-b5a4-fe4513047741 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:49.672383 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c674b4dc-71cd-4196-b5a4-fe4513047741 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:49.672860 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c674b4dc-71cd-4196-b5a4-fe4513047741 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:49.673746 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c674b4dc-71cd-4196-b5a4-fe4513047741 None admin] RBAC: Adding query filter params (name=member) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:27:49.674492 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c674b4dc-71cd-4196-b5a4-fe4513047741 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:49.676171 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c674b4dc-71cd-4196-b5a4-fe4513047741 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:49.682368 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:49.686674 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 76/148] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 15:27:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:49.693281 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c674b4dc-71cd-4196-b5a4-fe4513047741 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:49.785164 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf05991-ddf0-4640-8a57-e78e4556098a None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:49.813557 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:49.835753 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c3a7d2-9fe6-454f-a4dd-1eeb193f8cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:49.835977 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c674b4dc-71cd-4196-b5a4-fe4513047741 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:49.836386 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3a7d2-9fe6-454f-a4dd-1eeb193f8cd8 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:49.836664 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3a7d2-9fe6-454f-a4dd-1eeb193f8cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:49.836935 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3a7d2-9fe6-454f-a4dd-1eeb193f8cd8 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:49.837619 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c3a7d2-9fe6-454f-a4dd-1eeb193f8cd8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:27:49.837948 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c3a7d2-9fe6-454f-a4dd-1eeb193f8cd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:49.839428 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c3a7d2-9fe6-454f-a4dd-1eeb193f8cd8 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:49.857373 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 73/149] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:27:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:49.863215 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c3a7d2-9fe6-454f-a4dd-1eeb193f8cd8 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:49.875674 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a36bb15-7266-4570-84fe-d21204a15265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:49.876723 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36bb15-7266-4570-84fe-d21204a15265 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:49.877343 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36bb15-7266-4570-84fe-d21204a15265 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:49.877814 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36bb15-7266-4570-84fe-d21204a15265 None admin] PATH_INFO: `/v3/users/b0d4ab9befe041b08d614daa5c9e423c` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:49.879298 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a36bb15-7266-4570-84fe-d21204a15265 None admin] RBAC: Authorizing `identity:get_user(user_id=b0d4ab9befe041b08d614daa5c9e423c)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:49.881043 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a36bb15-7266-4570-84fe-d21204a15265 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:49.883172 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 77/150] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:27:49 2022] GET /identity/v3/users/b0d4ab9befe041b08d614daa5c9e423c => generated 275 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:49.889444 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a36bb15-7266-4570-84fe-d21204a15265 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:49.979516 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:50.035240 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c3a7d2-9fe6-454f-a4dd-1eeb193f8cd8 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:50.042096 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a36bb15-7266-4570-84fe-d21204a15265 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:50.072293 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa03e46b-be5a-4bda-a970-798f08066708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:50.073948 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03e46b-be5a-4bda-a970-798f08066708 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:50.074438 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03e46b-be5a-4bda-a970-798f08066708 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:50.074955 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03e46b-be5a-4bda-a970-798f08066708 None admin] PATH_INFO: `/v3/users/0fdbcf7ba1534603b18de5dbaf96574f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:50.076288 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa03e46b-be5a-4bda-a970-798f08066708 None admin] RBAC: Authorizing `identity:get_user(user_id=0fdbcf7ba1534603b18de5dbaf96574f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:50.077942 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa03e46b-be5a-4bda-a970-798f08066708 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:50.079365 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e2e29f-e073-4a04-b030-bc78c4bae95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:50.079960 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 74/151] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:27:49 2022] GET /identity/v3/users/0fdbcf7ba1534603b18de5dbaf96574f => generated 247 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:50.080319 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2e29f-e073-4a04-b030-bc78c4bae95a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:50.080606 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2e29f-e073-4a04-b030-bc78c4bae95a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:50.080965 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2e29f-e073-4a04-b030-bc78c4bae95a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:50.082025 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a6088-d6c8-4ae2-b1e7-3f4acc180a06 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:50.082494 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a6088-d6c8-4ae2-b1e7-3f4acc180a06 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:50.082961 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6a6088-d6c8-4ae2-b1e7-3f4acc180a06 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:50.084519 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 75/152] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:27:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:27:50.085466 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6a6088-d6c8-4ae2-b1e7-3f4acc180a06 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:50.148633 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:50.153805 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e2e29f-e073-4a04-b030-bc78c4bae95a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:50.155315 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e2e29f-e073-4a04-b030-bc78c4bae95a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:50.201294 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6a6088-d6c8-4ae2-b1e7-3f4acc180a06 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:50.213583 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:50.242794 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1190b2-6bc9-4691-8066-c43bc4be01ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:50.243745 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1190b2-6bc9-4691-8066-c43bc4be01ad None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:50.244217 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1190b2-6bc9-4691-8066-c43bc4be01ad None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:50.244677 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1190b2-6bc9-4691-8066-c43bc4be01ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:50.269753 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 78/153] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:27:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:27:50.275173 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70032589-e942-4803-aa6a-ccbad1d20185 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:50.275456 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70032589-e942-4803-aa6a-ccbad1d20185 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:50.275729 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70032589-e942-4803-aa6a-ccbad1d20185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:50.296840 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-70032589-e942-4803-aa6a-ccbad1d20185 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:27:50.297788 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-70032589-e942-4803-aa6a-ccbad1d20185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:27:50.348905 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1190b2-6bc9-4691-8066-c43bc4be01ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:50.350864 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1190b2-6bc9-4691-8066-c43bc4be01ad None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:50.389687 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 79/154] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:27:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:27:50.390868 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70032589-e942-4803-aa6a-ccbad1d20185 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:50.446243 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:50.475221 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 76/155] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:27:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:27:50.481024 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1190b2-6bc9-4691-8066-c43bc4be01ad None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:50.485455 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:50.507696 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70032589-e942-4803-aa6a-ccbad1d20185 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:50.554597 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1abfefa-5b21-4bf8-95c2-eb33e6db850d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:50.555254 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1abfefa-5b21-4bf8-95c2-eb33e6db850d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:50.555529 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1abfefa-5b21-4bf8-95c2-eb33e6db850d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:50.555802 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1abfefa-5b21-4bf8-95c2-eb33e6db850d None admin] PATH_INFO: `/v3/projects/4251d157536c4934bf6f66268f3037a9` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:50.556988 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1abfefa-5b21-4bf8-95c2-eb33e6db850d None admin] RBAC: Authorizing `identity:get_project(project_id=4251d157536c4934bf6f66268f3037a9)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:50.559014 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1abfefa-5b21-4bf8-95c2-eb33e6db850d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:50.560660 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 80/156] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:27:50 2022] GET /identity/v3/projects/4251d157536c4934bf6f66268f3037a9 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:50.566749 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7607f17d-6762-4be2-922e-5bae211f63bf None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:50.567019 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7607f17d-6762-4be2-922e-5bae211f63bf None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:50.567293 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7607f17d-6762-4be2-922e-5bae211f63bf None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:50.568600 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 81/157] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:27:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:27:50.570483 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7607f17d-6762-4be2-922e-5bae211f63bf None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:50.598384 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1190b2-6bc9-4691-8066-c43bc4be01ad None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:50.639619 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d50bd-194c-4351-86d2-c1d31e5bbf6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:50.640803 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d50bd-194c-4351-86d2-c1d31e5bbf6d None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:50.641348 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d50bd-194c-4351-86d2-c1d31e5bbf6d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:50.641816 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d50bd-194c-4351-86d2-c1d31e5bbf6d None admin] PATH_INFO: `/v3/projects/4251d157536c4934bf6f66268f3037a9` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:50.643345 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3d50bd-194c-4351-86d2-c1d31e5bbf6d None admin] RBAC: Authorizing `identity:get_project(project_id=4251d157536c4934bf6f66268f3037a9)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:50.644912 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3d50bd-194c-4351-86d2-c1d31e5bbf6d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:50.651031 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 77/158] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:27:50 2022] GET /identity/v3/projects/4251d157536c4934bf6f66268f3037a9 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:27:50.651031 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d758af-eff2-4eef-9493-ab25341bd69f None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:50.651031 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d758af-eff2-4eef-9493-ab25341bd69f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:50.651031 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d758af-eff2-4eef-9493-ab25341bd69f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:50.669233 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-75d758af-eff2-4eef-9493-ab25341bd69f None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:27:50.670589 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-75d758af-eff2-4eef-9493-ab25341bd69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:27:50.696998 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:50.724338 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7607f17d-6762-4be2-922e-5bae211f63bf None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:50.774421 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5847cd8f-952d-46a1-aaeb-a97b00b153d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:50.774965 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847cd8f-952d-46a1-aaeb-a97b00b153d0 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:50.775339 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847cd8f-952d-46a1-aaeb-a97b00b153d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:50.775619 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5847cd8f-952d-46a1-aaeb-a97b00b153d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:50.776445 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5847cd8f-952d-46a1-aaeb-a97b00b153d0 None admin] RBAC: Adding query filter params (role.id=e152a36b5a1b470b912b1d705462f512, scope.project.id=4251d157536c4934bf6f66268f3037a9, user.id=b0d4ab9befe041b08d614daa5c9e423c) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:27:50.777388 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5847cd8f-952d-46a1-aaeb-a97b00b153d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:50.778784 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 78/159] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:27:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:27:50.779255 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5847cd8f-952d-46a1-aaeb-a97b00b153d0 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:50.782952 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a6dee-55cf-46d3-aac7-2a016ea89c5c None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:50.783554 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a6dee-55cf-46d3-aac7-2a016ea89c5c None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:50.784021 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a6dee-55cf-46d3-aac7-2a016ea89c5c None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:50.785481 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 79/160] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:27:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:27:50.786395 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265a6dee-55cf-46d3-aac7-2a016ea89c5c None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:50.794754 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 82/161] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 15:27:50 2022] GET /identity/v3/role_assignments?user.id=b0d4ab9befe041b08d614daa5c9e423c&scope.project.id=4251d157536c4934bf6f66268f3037a9&role.id=e152a36b5a1b470b912b1d705462f512 => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:50.795476 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db59f2-c4b7-453d-b150-24c919124272 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:50.795763 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db59f2-c4b7-453d-b150-24c919124272 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:50.796037 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db59f2-c4b7-453d-b150-24c919124272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:50.821321 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-68db59f2-c4b7-453d-b150-24c919124272 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:27:50.823161 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-68db59f2-c4b7-453d-b150-24c919124272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:27:50.836348 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:50.910533 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 83/162] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:27:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:27:50.911745 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68db59f2-c4b7-453d-b150-24c919124272 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:51.059937 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68db59f2-c4b7-453d-b150-24c919124272 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:51.067458 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265a6dee-55cf-46d3-aac7-2a016ea89c5c None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:51.112661 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5ff936-5593-4677-bf05-910219d7b9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:51.113658 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ff936-5593-4677-bf05-910219d7b9d1 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:51.114168 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ff936-5593-4677-bf05-910219d7b9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:51.114629 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ff936-5593-4677-bf05-910219d7b9d1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:51.124088 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daad51c-ad30-40fd-8e47-9a13ae4e8779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:51.125006 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daad51c-ad30-40fd-8e47-9a13ae4e8779 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:51.125467 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daad51c-ad30-40fd-8e47-9a13ae4e8779 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:51.125968 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daad51c-ad30-40fd-8e47-9a13ae4e8779 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:51.126881 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daad51c-ad30-40fd-8e47-9a13ae4e8779 None admin] RBAC: Adding query filter params (role.id=62d96ac9513c452894fc67b442cfc59f, scope.project.id=4251d157536c4934bf6f66268f3037a9, user.id=0fdbcf7ba1534603b18de5dbaf96574f) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:27:51.128455 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daad51c-ad30-40fd-8e47-9a13ae4e8779 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:51.130131 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daad51c-ad30-40fd-8e47-9a13ae4e8779 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:51.136964 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5ff936-5593-4677-bf05-910219d7b9d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:27:51.142110 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-5f5ff936-5593-4677-bf05-910219d7b9d1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:27:51.142110 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:27:51.142110 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:27:51.142110 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:27:51.142110 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:27:51.142110 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:27:51.142110 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:27:51.142110 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:27:51.142110 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:27:51.142110 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:27:51.142110 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:27:51.142110 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:27:51.142110 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:27:51.142110 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:27:51.142110 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:27:51.142110 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:27:51.142957 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:27:51.142957 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:27:51.142957 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:27:51.142957 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:27:51.142957 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:27:51.142957 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:27:51.142957 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:27:51.142957 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:27:51.142957 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:27:51.142957 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:27:51.142957 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:27:51.142957 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:27:51.142957 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:27:51.142957 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:27:51.142957 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:27:51.142957 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:27:51.143808 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:27:51.143808 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:27:51.143808 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:27:51.143808 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:27:51.143808 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:27:51.143808 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:27:51.143808 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:27:51.143808 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:27:51.143808 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:27:51.143808 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:27:51.143808 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:27:51.143808 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:27:51.148020 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 80/163] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 15:27:50 2022] GET /identity/v3/role_assignments?user.id=0fdbcf7ba1534603b18de5dbaf96574f&scope.project.id=4251d157536c4934bf6f66268f3037a9&role.id=62d96ac9513c452894fc67b442cfc59f => generated 608 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:51.149287 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 84/164] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 15:27:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:27:51.155959 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daad51c-ad30-40fd-8e47-9a13ae4e8779 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:51.182042 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:51.243223 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5ff936-5593-4677-bf05-910219d7b9d1 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:51.251062 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:51.296107 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daad51c-ad30-40fd-8e47-9a13ae4e8779 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:51.355165 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95e90f-ac1c-46f4-b84d-69d638c3b0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:51.355704 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95e90f-ac1c-46f4-b84d-69d638c3b0c4 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:51.355979 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95e90f-ac1c-46f4-b84d-69d638c3b0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:51.356243 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95e90f-ac1c-46f4-b84d-69d638c3b0c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:51.356917 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c95e90f-ac1c-46f4-b84d-69d638c3b0c4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:27:51.357219 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c95e90f-ac1c-46f4-b84d-69d638c3b0c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:51.358839 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c95e90f-ac1c-46f4-b84d-69d638c3b0c4 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:51.373012 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 81/165] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:27:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:51.403418 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95e90f-ac1c-46f4-b84d-69d638c3b0c4 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:51.414879 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5ff936-5593-4677-bf05-910219d7b9d1 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:51.451461 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bc9397-429e-4247-a0e0-ff878314a30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:51.452115 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc9397-429e-4247-a0e0-ff878314a30a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:51.452396 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc9397-429e-4247-a0e0-ff878314a30a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:51.452673 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc9397-429e-4247-a0e0-ff878314a30a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:51.467909 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bc9397-429e-4247-a0e0-ff878314a30a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:27:51.472906 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-b4bc9397-429e-4247-a0e0-ff878314a30a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 15:27:51.472906 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:27:51.472906 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:27:51.472906 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:27:51.472906 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:27:51.472906 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:27:51.472906 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:27:51.472906 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:27:51.472906 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:27:51.472906 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:27:51.472906 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:27:51.472906 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:27:51.472906 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:27:51.472906 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:27:51.472906 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:27:51.472906 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:27:51.473784 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:27:51.473784 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:27:51.473784 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:27:51.473784 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:27:51.473784 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:27:51.473784 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:27:51.473784 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:27:51.473784 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:27:51.473784 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:27:51.473784 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:27:51.473784 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:27:51.473784 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:27:51.473784 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:27:51.473784 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:27:51.473784 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:27:51.473784 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:27:51.474824 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:27:51.474824 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:27:51.474824 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:27:51.474824 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:27:51.474824 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:27:51.474824 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:27:51.474824 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:27:51.474824 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:27:51.474824 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:27:51.474824 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:27:51.474824 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 15:27:51.474824 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:27:51.474824 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 85/166] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:27:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:27:51.480519 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bc9397-429e-4247-a0e0-ff878314a30a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:51.551340 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:51.557242 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:51.589088 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95e90f-ac1c-46f4-b84d-69d638c3b0c4 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:51.599058 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bc9397-429e-4247-a0e0-ff878314a30a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:51.638348 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6673f-2357-4ce9-aef7-cfcb869b6734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:51.639186 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6673f-2357-4ce9-aef7-cfcb869b6734 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:51.639473 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6673f-2357-4ce9-aef7-cfcb869b6734 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:51.639750 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6673f-2357-4ce9-aef7-cfcb869b6734 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:51.640423 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb6673f-2357-4ce9-aef7-cfcb869b6734 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:27:51.640726 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb6673f-2357-4ce9-aef7-cfcb869b6734 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:51.642629 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb6673f-2357-4ce9-aef7-cfcb869b6734 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:51.651623 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 86/167] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 15:27:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:27:51.657250 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6673f-2357-4ce9-aef7-cfcb869b6734 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:51.689615 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c43c66e-6ac0-4d9f-850a-f95e083afd77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:51.690297 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c43c66e-6ac0-4d9f-850a-f95e083afd77 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:51.690575 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c43c66e-6ac0-4d9f-850a-f95e083afd77 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:51.690895 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c43c66e-6ac0-4d9f-850a-f95e083afd77 None admin] PATH_INFO: `/v3/users/b0d4ab9befe041b08d614daa5c9e423c` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:51.694641 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c43c66e-6ac0-4d9f-850a-f95e083afd77 None admin] RBAC: Authorizing `identity:get_user(user_id=b0d4ab9befe041b08d614daa5c9e423c)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:51.696201 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c43c66e-6ac0-4d9f-850a-f95e083afd77 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:51.698013 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 82/168] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:27:51 2022] GET /identity/v3/users/b0d4ab9befe041b08d614daa5c9e423c => generated 275 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:51.702510 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c43c66e-6ac0-4d9f-850a-f95e083afd77 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:51.777828 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6673f-2357-4ce9-aef7-cfcb869b6734 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:51.810885 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:51.819288 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817bbbd2-5f55-4aa1-a4c9-a1f48f897884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:51.820140 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817bbbd2-5f55-4aa1-a4c9-a1f48f897884 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:51.820421 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817bbbd2-5f55-4aa1-a4c9-a1f48f897884 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:51.820693 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817bbbd2-5f55-4aa1-a4c9-a1f48f897884 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:51.833703 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817bbbd2-5f55-4aa1-a4c9-a1f48f897884 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:27:51.839091 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-817bbbd2-5f55-4aa1-a4c9-a1f48f897884 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:27:51.839091 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:27:51.839091 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:27:51.839091 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:27:51.839091 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:27:51.839091 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:27:51.839091 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:27:51.839091 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:27:51.839091 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:27:51.839091 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:27:51.839091 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:27:51.839091 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:27:51.839091 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:27:51.839091 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:27:51.839091 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:27:51.839091 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:27:51.840240 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:27:51.840240 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:27:51.840240 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:27:51.840240 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:27:51.840240 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:27:51.840240 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:27:51.840240 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:27:51.840240 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:27:51.840240 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:27:51.840240 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:27:51.840240 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:27:51.840240 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:27:51.840240 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:27:51.840240 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:27:51.840240 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:27:51.840240 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:27:51.841647 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:27:51.841647 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:27:51.841647 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:27:51.841647 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:27:51.841647 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:27:51.841647 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:27:51.841647 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:27:51.841647 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:27:51.841647 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:27:51.841647 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:27:51.841647 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:27:51.841647 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:27:51.841647 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 87/169] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:27:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:27:51.841647 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817bbbd2-5f55-4aa1-a4c9-a1f48f897884 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:51.954646 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817bbbd2-5f55-4aa1-a4c9-a1f48f897884 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:51.994389 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6270c822-ddcf-4fda-a48b-6ae6ed843b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:51.995251 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6270c822-ddcf-4fda-a48b-6ae6ed843b55 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:51.995540 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6270c822-ddcf-4fda-a48b-6ae6ed843b55 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:51.995816 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6270c822-ddcf-4fda-a48b-6ae6ed843b55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:52.004834 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c43c66e-6ac0-4d9f-850a-f95e083afd77 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:52.058958 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:52.073725 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6270c822-ddcf-4fda-a48b-6ae6ed843b55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:52.076100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6270c822-ddcf-4fda-a48b-6ae6ed843b55 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:52.110594 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd97dc17-0f07-45d3-a9f7-c03a934a1e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:52.111242 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97dc17-0f07-45d3-a9f7-c03a934a1e7f None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:52.111521 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97dc17-0f07-45d3-a9f7-c03a934a1e7f None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:52.111793 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97dc17-0f07-45d3-a9f7-c03a934a1e7f None admin] PATH_INFO: `/v3/users/9f3930cf37644c92bae454a9ddcc1670` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:52.117246 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd97dc17-0f07-45d3-a9f7-c03a934a1e7f None admin] RBAC: Authorizing `identity:get_user(user_id=9f3930cf37644c92bae454a9ddcc1670)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:52.118837 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd97dc17-0f07-45d3-a9f7-c03a934a1e7f None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:52.120666 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 83/170] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:27:51 2022] GET /identity/v3/users/9f3930cf37644c92bae454a9ddcc1670 => generated 289 bytes in 422 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:52.121252 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd97dc17-0f07-45d3-a9f7-c03a934a1e7f None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:52.164480 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:52.192804 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 88/171] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:27:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:27:52.193676 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6270c822-ddcf-4fda-a48b-6ae6ed843b55 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:52.335464 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd97dc17-0f07-45d3-a9f7-c03a934a1e7f None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:52.350198 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:52.361921 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6270c822-ddcf-4fda-a48b-6ae6ed843b55 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:52.408433 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93453429-abcb-4bc9-9eb5-d8a50e547c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:52.409229 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5237ac4-1437-437c-b23a-0223d6a6aef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:52.409995 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93453429-abcb-4bc9-9eb5-d8a50e547c2d None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:52.410705 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5237ac4-1437-437c-b23a-0223d6a6aef5 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:52.411076 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93453429-abcb-4bc9-9eb5-d8a50e547c2d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:52.411446 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5237ac4-1437-437c-b23a-0223d6a6aef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:52.411811 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93453429-abcb-4bc9-9eb5-d8a50e547c2d None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:52.412190 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5237ac4-1437-437c-b23a-0223d6a6aef5 None admin] PATH_INFO: `/v3/projects/4251d157536c4934bf6f66268f3037a9` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:52.412946 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93453429-abcb-4bc9-9eb5-d8a50e547c2d None admin] RBAC: Adding query filter params (name=member) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:27:52.414189 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5237ac4-1437-437c-b23a-0223d6a6aef5 None admin] RBAC: Authorizing `identity:get_project(project_id=4251d157536c4934bf6f66268f3037a9)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:52.414676 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93453429-abcb-4bc9-9eb5-d8a50e547c2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:52.416118 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93453429-abcb-4bc9-9eb5-d8a50e547c2d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:52.418238 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5237ac4-1437-437c-b23a-0223d6a6aef5 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:52.422264 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 89/172] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:27:52 2022] GET /identity/v3/projects/4251d157536c4934bf6f66268f3037a9 => generated 298 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:52.425689 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5237ac4-1437-437c-b23a-0223d6a6aef5 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:52.437010 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 84/173] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 15:27:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:52.437799 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93453429-abcb-4bc9-9eb5-d8a50e547c2d None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:52.548085 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:52.580691 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93453429-abcb-4bc9-9eb5-d8a50e547c2d None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:52.628592 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5237ac4-1437-437c-b23a-0223d6a6aef5 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:52.653733 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2252dd4-b0f1-4b04-b56e-32531133e6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:52.654428 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2252dd4-b0f1-4b04-b56e-32531133e6c0 None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:52.654702 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2252dd4-b0f1-4b04-b56e-32531133e6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:52.654973 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2252dd4-b0f1-4b04-b56e-32531133e6c0 None admin] PATH_INFO: `/v3/projects/4251d157536c4934bf6f66268f3037a9/users/9f3930cf37644c92bae454a9ddcc1670/roles/9e4cabcdb1e349c699da538ce00d1396` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:52.659484 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2252dd4-b0f1-4b04-b56e-32531133e6c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=4251d157536c4934bf6f66268f3037a9, user_id=9f3930cf37644c92bae454a9ddcc1670, role_id=9e4cabcdb1e349c699da538ce00d1396)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:52.660862 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2252dd4-b0f1-4b04-b56e-32531133e6c0 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:52.685223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 85/174] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 15:27:52 2022] PUT /identity/v3/projects/4251d157536c4934bf6f66268f3037a9/users/9f3930cf37644c92bae454a9ddcc1670/roles/9e4cabcdb1e349c699da538ce00d1396 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:27:52.686091 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2252dd4-b0f1-4b04-b56e-32531133e6c0 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:52.697164 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f13a91a-b53f-46ea-96e5-ecddcd99b86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:52.697821 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f13a91a-b53f-46ea-96e5-ecddcd99b86e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:52.698121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f13a91a-b53f-46ea-96e5-ecddcd99b86e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:52.698588 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f13a91a-b53f-46ea-96e5-ecddcd99b86e None admin] PATH_INFO: `/v3/projects/4251d157536c4934bf6f66268f3037a9` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:52.703637 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f13a91a-b53f-46ea-96e5-ecddcd99b86e None admin] RBAC: Authorizing `identity:get_project(project_id=4251d157536c4934bf6f66268f3037a9)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:52.705491 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f13a91a-b53f-46ea-96e5-ecddcd99b86e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:52.710473 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 90/175] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:27:52 2022] GET /identity/v3/projects/4251d157536c4934bf6f66268f3037a9 => generated 298 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:52.720683 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f13a91a-b53f-46ea-96e5-ecddcd99b86e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:52.766580 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:52.896195 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2252dd4-b0f1-4b04-b56e-32531133e6c0 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:52.916744 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f13a91a-b53f-46ea-96e5-ecddcd99b86e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:52.954990 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b81a3da-4449-486a-9b5c-c38e4e6b4c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:52.955624 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b81a3da-4449-486a-9b5c-c38e4e6b4c3c None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:52.955918 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b81a3da-4449-486a-9b5c-c38e4e6b4c3c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:52.957883 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b81a3da-4449-486a-9b5c-c38e4e6b4c3c None admin] PATH_INFO: `/v3/users/b0d4ab9befe041b08d614daa5c9e423c` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:52.957883 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b81a3da-4449-486a-9b5c-c38e4e6b4c3c None admin] RBAC: Authorizing `identity:get_user(user_id=b0d4ab9befe041b08d614daa5c9e423c)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:52.961064 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b81a3da-4449-486a-9b5c-c38e4e6b4c3c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:52.963011 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 86/176] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:27:52 2022] GET /identity/v3/users/b0d4ab9befe041b08d614daa5c9e423c => generated 275 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:52.967828 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fcc1df-58db-415a-bf6f-1851fc137b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:52.968453 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fcc1df-58db-415a-bf6f-1851fc137b1f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:52.968738 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fcc1df-58db-415a-bf6f-1851fc137b1f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:52.969089 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fcc1df-58db-415a-bf6f-1851fc137b1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:52.969829 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fcc1df-58db-415a-bf6f-1851fc137b1f None admin] RBAC: Adding query filter params (role.id=5038c8140b7145ceb7494210a5c9f9f0, scope.project.id=4251d157536c4934bf6f66268f3037a9, user.id=b0d4ab9befe041b08d614daa5c9e423c) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:27:52.970353 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fcc1df-58db-415a-bf6f-1851fc137b1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:52.971351 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b81a3da-4449-486a-9b5c-c38e4e6b4c3c None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:52.971595 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fcc1df-58db-415a-bf6f-1851fc137b1f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:52.983501 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:52.984613 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 91/177] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 15:27:52 2022] GET /identity/v3/role_assignments?user.id=b0d4ab9befe041b08d614daa5c9e423c&scope.project.id=4251d157536c4934bf6f66268f3037a9&role.id=5038c8140b7145ceb7494210a5c9f9f0 => generated 608 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:27:53.108553 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b81a3da-4449-486a-9b5c-c38e4e6b4c3c None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:53.159587 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc5ebaa-60eb-460e-a9db-8294006257e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:53.160492 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc5ebaa-60eb-460e-a9db-8294006257e1 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:53.160915 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc5ebaa-60eb-460e-a9db-8294006257e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:53.161347 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc5ebaa-60eb-460e-a9db-8294006257e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:53.241895 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da8250-d115-4e21-b246-3704f10cfe5a None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:53.241895 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da8250-d115-4e21-b246-3704f10cfe5a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:53.241895 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da8250-d115-4e21-b246-3704f10cfe5a None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:53.242346 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 92/178] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:27:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:27:53.254092 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10118067-068a-4770-a53e-b5c435ac6787 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:53.254376 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10118067-068a-4770-a53e-b5c435ac6787 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:53.254654 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10118067-068a-4770-a53e-b5c435ac6787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:53.263294 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc5ebaa-60eb-460e-a9db-8294006257e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:53.264888 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc5ebaa-60eb-460e-a9db-8294006257e1 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:53.279200 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:53.311628 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-10118067-068a-4770-a53e-b5c435ac6787 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:27:53.313317 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-10118067-068a-4770-a53e-b5c435ac6787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:27:53.421161 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 87/179] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:27:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:27:53.427371 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc5ebaa-60eb-460e-a9db-8294006257e1 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:53.478031 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 93/180] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:27:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:27:53.554044 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc5ebaa-60eb-460e-a9db-8294006257e1 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:53.562389 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:53.591546 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f0299d-a7e2-4f80-abb7-cfd58b1510ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:53.592232 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0299d-a7e2-4f80-abb7-cfd58b1510ac None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:53.592511 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0299d-a7e2-4f80-abb7-cfd58b1510ac None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:53.592780 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0299d-a7e2-4f80-abb7-cfd58b1510ac None admin] PATH_INFO: `/v3/projects/17a69146ef934856b8533fdb8ddfd873` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:53.593940 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f0299d-a7e2-4f80-abb7-cfd58b1510ac None admin] RBAC: Authorizing `identity:get_project(project_id=17a69146ef934856b8533fdb8ddfd873)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:53.595386 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f0299d-a7e2-4f80-abb7-cfd58b1510ac None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:53.597172 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 88/181] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:27:53 2022] GET /identity/v3/projects/17a69146ef934856b8533fdb8ddfd873 => generated 312 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:27:53.603046 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10118067-068a-4770-a53e-b5c435ac6787 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:53.621597 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:53.730983 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10118067-068a-4770-a53e-b5c435ac6787 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:53.773177 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f3539-4c46-4283-9670-192e88ef78f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:53.773780 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f3539-4c46-4283-9670-192e88ef78f9 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:53.774111 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f3539-4c46-4283-9670-192e88ef78f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:53.774380 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f3539-4c46-4283-9670-192e88ef78f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:53.775100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f3539-4c46-4283-9670-192e88ef78f9 None admin] RBAC: Adding query filter params (role.id=e152a36b5a1b470b912b1d705462f512, scope.project.id=17a69146ef934856b8533fdb8ddfd873, user.id=b0d4ab9befe041b08d614daa5c9e423c) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:27:53.775405 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f3539-4c46-4283-9670-192e88ef78f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:53.777413 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f3539-4c46-4283-9670-192e88ef78f9 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:53.786713 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 94/182] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 15:27:53 2022] GET /identity/v3/role_assignments?user.id=b0d4ab9befe041b08d614daa5c9e423c&scope.project.id=17a69146ef934856b8533fdb8ddfd873&role.id=e152a36b5a1b470b912b1d705462f512 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:53.830757 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5873d86-0c05-4deb-bf9a-2b07f491aba8 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:53.831076 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5873d86-0c05-4deb-bf9a-2b07f491aba8 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:53.831308 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5873d86-0c05-4deb-bf9a-2b07f491aba8 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:53.832639 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 89/183] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:27:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:27:53.838323 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a766c4f-8ed7-46aa-9a51-aad96e98aba4 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:53.838612 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a766c4f-8ed7-46aa-9a51-aad96e98aba4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:53.838917 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a766c4f-8ed7-46aa-9a51-aad96e98aba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:53.860188 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-6a766c4f-8ed7-46aa-9a51-aad96e98aba4 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:27:53.861374 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6a766c4f-8ed7-46aa-9a51-aad96e98aba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:27:53.953337 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 95/184] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:27:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:27:54.058941 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5873d86-0c05-4deb-bf9a-2b07f491aba8 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:54.080112 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e86c17-cc3b-4032-b712-7e0b353819fe None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:54.080412 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e86c17-cc3b-4032-b712-7e0b353819fe None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:54.080647 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e86c17-cc3b-4032-b712-7e0b353819fe None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:54.082167 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 96/185] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:27:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:27:54.087013 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f8482-fb7b-44b0-bfbd-70c964918fa3 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:54.087296 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f8482-fb7b-44b0-bfbd-70c964918fa3 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:54.087572 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f8482-fb7b-44b0-bfbd-70c964918fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:54.110961 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-3f5f8482-fb7b-44b0-bfbd-70c964918fa3 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:27:54.112134 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3f5f8482-fb7b-44b0-bfbd-70c964918fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:27:54.116706 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:54.190998 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 97/186] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:27:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:27:54.205114 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5873d86-0c05-4deb-bf9a-2b07f491aba8 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:54.246102 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868a63dd-ca41-4df5-a4a0-60ef57ac39fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:54.246102 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a63dd-ca41-4df5-a4a0-60ef57ac39fb None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:54.246102 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a63dd-ca41-4df5-a4a0-60ef57ac39fb None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:54.246102 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a63dd-ca41-4df5-a4a0-60ef57ac39fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:54.257633 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868a63dd-ca41-4df5-a4a0-60ef57ac39fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:27:54.262841 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-868a63dd-ca41-4df5-a4a0-60ef57ac39fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:27:54.262841 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:27:54.262841 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:27:54.262841 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:27:54.262841 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:27:54.262841 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:27:54.262841 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:27:54.262841 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:27:54.262841 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:27:54.262841 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:27:54.262841 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:27:54.262841 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:27:54.262841 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:27:54.262841 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:27:54.262841 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:27:54.262841 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:27:54.263716 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:27:54.263716 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:27:54.263716 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:27:54.263716 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:27:54.263716 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:27:54.263716 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:27:54.263716 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:27:54.263716 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:27:54.263716 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:27:54.263716 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:27:54.263716 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:27:54.263716 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:27:54.263716 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:27:54.263716 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:27:54.263716 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:27:54.263716 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:27:54.264480 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:27:54.264480 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:27:54.264480 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:27:54.264480 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:27:54.264480 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:27:54.264480 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:27:54.264480 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:27:54.264480 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:27:54.264480 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:27:54.264480 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:27:54.264480 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:27:54.264480 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:27:54.266850 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 90/187] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 15:27:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:27:54.272994 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5f8482-fb7b-44b0-bfbd-70c964918fa3 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:54.274120 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868a63dd-ca41-4df5-a4a0-60ef57ac39fb None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:54.322215 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:54.405795 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868a63dd-ca41-4df5-a4a0-60ef57ac39fb None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:54.427203 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5f8482-fb7b-44b0-bfbd-70c964918fa3 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:54.446310 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db7332d-1c1b-48d4-bdea-361a509d7431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:54.446956 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db7332d-1c1b-48d4-bdea-361a509d7431 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:54.447275 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db7332d-1c1b-48d4-bdea-361a509d7431 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:54.447440 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db7332d-1c1b-48d4-bdea-361a509d7431 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:54.448107 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db7332d-1c1b-48d4-bdea-361a509d7431 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:27:54.448445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db7332d-1c1b-48d4-bdea-361a509d7431 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:54.449821 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db7332d-1c1b-48d4-bdea-361a509d7431 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:54.458397 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 91/188] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:27:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:54.464068 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db7332d-1c1b-48d4-bdea-361a509d7431 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:54.472886 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b93305d-7684-4c59-b905-ac05609866ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:54.474349 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93305d-7684-4c59-b905-ac05609866ee None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:54.474825 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93305d-7684-4c59-b905-ac05609866ee None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:54.475301 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93305d-7684-4c59-b905-ac05609866ee None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:54.486267 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:54.490530 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b93305d-7684-4c59-b905-ac05609866ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:27:54.496133 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-3b93305d-7684-4c59-b905-ac05609866ee None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:27:54.496133 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:27:54.496133 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:27:54.496133 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:27:54.496133 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:27:54.496133 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:27:54.496133 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:27:54.496133 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:27:54.496133 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:27:54.496133 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:27:54.496133 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:27:54.496133 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:27:54.496133 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:27:54.496133 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:27:54.496133 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:27:54.496133 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:27:54.497202 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:27:54.497202 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:27:54.497202 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:27:54.497202 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:27:54.497202 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:27:54.497202 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:27:54.497202 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:27:54.497202 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:27:54.497202 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:27:54.497202 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:27:54.497202 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:27:54.497202 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:27:54.497202 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:27:54.497202 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:27:54.497202 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:27:54.497202 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:27:54.498077 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:27:54.498077 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:27:54.498077 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:27:54.498077 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:27:54.498077 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:27:54.498077 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:27:54.498077 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:27:54.498077 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:27:54.498077 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:27:54.498077 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:27:54.498077 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:27:54.498077 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:27:54.499178 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 98/189] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 15:27:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:27:54.499664 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b93305d-7684-4c59-b905-ac05609866ee None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:54.581483 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:54.587521 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db7332d-1c1b-48d4-bdea-361a509d7431 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:54.597626 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b93305d-7684-4c59-b905-ac05609866ee None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:54.622968 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f65607c-6ace-4265-b458-215b451ba8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:54.623593 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f65607c-6ace-4265-b458-215b451ba8f0 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:54.623865 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f65607c-6ace-4265-b458-215b451ba8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:54.624131 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f65607c-6ace-4265-b458-215b451ba8f0 None admin] PATH_INFO: `/v3/users/91c065b6fe104421a64d9eb474a7761c` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:54.632421 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f213fd8f-472f-4eac-bb8f-993e5955e81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:54.633039 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f213fd8f-472f-4eac-bb8f-993e5955e81e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:54.633314 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f213fd8f-472f-4eac-bb8f-993e5955e81e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:54.633580 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f213fd8f-472f-4eac-bb8f-993e5955e81e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:54.634692 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f213fd8f-472f-4eac-bb8f-993e5955e81e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:54.636185 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f213fd8f-472f-4eac-bb8f-993e5955e81e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:54.642438 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 99/190] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:27:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:54.642573 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f65607c-6ace-4265-b458-215b451ba8f0 None admin] RBAC: Authorizing `identity:get_user(user_id=91c065b6fe104421a64d9eb474a7761c)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:54.642665 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f213fd8f-472f-4eac-bb8f-993e5955e81e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:54.642768 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f65607c-6ace-4265-b458-215b451ba8f0 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:54.642768 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 92/191] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:27:54 2022] GET /identity/v3/users/91c065b6fe104421a64d9eb474a7761c => generated 283 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:27:54.643845 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f65607c-6ace-4265-b458-215b451ba8f0 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:54.708224 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:54.741619 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f65607c-6ace-4265-b458-215b451ba8f0 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:54.743878 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f213fd8f-472f-4eac-bb8f-993e5955e81e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:54.777959 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324def07-a31f-4378-b5cf-a8d3baf679f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:54.778450 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf7d21-72a0-4799-a49a-4d6595c5c8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:54.778562 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324def07-a31f-4378-b5cf-a8d3baf679f4 None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:54.778819 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324def07-a31f-4378-b5cf-a8d3baf679f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:54.779051 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf7d21-72a0-4799-a49a-4d6595c5c8cf None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:54.779144 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324def07-a31f-4378-b5cf-a8d3baf679f4 None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:54.779333 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf7d21-72a0-4799-a49a-4d6595c5c8cf None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:54.779604 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf7d21-72a0-4799-a49a-4d6595c5c8cf None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:54.779692 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324def07-a31f-4378-b5cf-a8d3baf679f4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:54.780257 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cf7d21-72a0-4799-a49a-4d6595c5c8cf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:27:54.780553 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cf7d21-72a0-4799-a49a-4d6595c5c8cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:54.781097 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324def07-a31f-4378-b5cf-a8d3baf679f4 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:54.781932 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cf7d21-72a0-4799-a49a-4d6595c5c8cf None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:54.789843 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 100/192] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:27:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:54.795831 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf7d21-72a0-4799-a49a-4d6595c5c8cf None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:54.804052 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 93/193] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 15:27:54 2022] POST /identity/v3/users => generated 297 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:27:54.805064 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324def07-a31f-4378-b5cf-a8d3baf679f4 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:54.826675 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:54.928154 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf7d21-72a0-4799-a49a-4d6595c5c8cf None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:54.951505 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324def07-a31f-4378-b5cf-a8d3baf679f4 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:54.963585 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:54.970831 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b4fb2e-b6cf-4e8a-9d26-e863d98d7ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:54.971756 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b4fb2e-b6cf-4e8a-9d26-e863d98d7ab1 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:54.972536 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b4fb2e-b6cf-4e8a-9d26-e863d98d7ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:54.973668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b4fb2e-b6cf-4e8a-9d26-e863d98d7ab1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:54.997247 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35219ad4-5452-44a8-aabc-d7020375a492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:54.998083 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35219ad4-5452-44a8-aabc-d7020375a492 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:54.998451 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35219ad4-5452-44a8-aabc-d7020375a492 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:54.998849 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35219ad4-5452-44a8-aabc-d7020375a492 None admin] PATH_INFO: `/v3/users/91c065b6fe104421a64d9eb474a7761c` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:55.000149 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35219ad4-5452-44a8-aabc-d7020375a492 None admin] RBAC: Authorizing `identity:get_user(user_id=91c065b6fe104421a64d9eb474a7761c)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:55.001626 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35219ad4-5452-44a8-aabc-d7020375a492 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:55.003264 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 94/194] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:27:54 2022] GET /identity/v3/users/91c065b6fe104421a64d9eb474a7761c => generated 283 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:55.008996 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35219ad4-5452-44a8-aabc-d7020375a492 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:55.054855 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b4fb2e-b6cf-4e8a-9d26-e863d98d7ab1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:55.056357 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b4fb2e-b6cf-4e8a-9d26-e863d98d7ab1 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:55.114641 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:55.144974 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35219ad4-5452-44a8-aabc-d7020375a492 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:55.177686 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 101/195] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:27:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:27:55.183575 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b4fb2e-b6cf-4e8a-9d26-e863d98d7ab1 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:55.194066 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801c50d0-1e76-44eb-933e-bded0285272b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:55.194774 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c50d0-1e76-44eb-933e-bded0285272b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:55.195119 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c50d0-1e76-44eb-933e-bded0285272b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:55.195459 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c50d0-1e76-44eb-933e-bded0285272b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:55.219348 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:55.320872 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b4fb2e-b6cf-4e8a-9d26-e863d98d7ab1 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:55.337284 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801c50d0-1e76-44eb-933e-bded0285272b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:55.343047 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801c50d0-1e76-44eb-933e-bded0285272b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:55.369908 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcf5fb1-3002-4c8d-8c96-c148bc9a08fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:55.372088 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcf5fb1-3002-4c8d-8c96-c148bc9a08fb None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:55.372738 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcf5fb1-3002-4c8d-8c96-c148bc9a08fb None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:55.373272 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcf5fb1-3002-4c8d-8c96-c148bc9a08fb None admin] PATH_INFO: `/v3/projects/fbb3348b420b4f6ba5d171c387e5a403` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:55.374694 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcf5fb1-3002-4c8d-8c96-c148bc9a08fb None admin] RBAC: Authorizing `identity:get_project(project_id=fbb3348b420b4f6ba5d171c387e5a403)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:55.376359 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcf5fb1-3002-4c8d-8c96-c148bc9a08fb None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:55.378585 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 102/196] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:27:55 2022] GET /identity/v3/projects/fbb3348b420b4f6ba5d171c387e5a403 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:27:55.390909 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcf5fb1-3002-4c8d-8c96-c148bc9a08fb None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:55.456874 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:55.483032 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 95/197] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:27:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 28 15:27:55.489279 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5109a33-fe1b-4184-bfb7-f33a0278b693 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:55.489805 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5109a33-fe1b-4184-bfb7-f33a0278b693 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:55.490332 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5109a33-fe1b-4184-bfb7-f33a0278b693 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:55.491879 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 96/198] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:27:55 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:27:55.497660 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5109a33-fe1b-4184-bfb7-f33a0278b693 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:55.515334 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:55.519026 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcf5fb1-3002-4c8d-8c96-c148bc9a08fb None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:55.554862 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e4a0df-d0fe-4f75-807e-bb964f55e505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:55.555526 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e4a0df-d0fe-4f75-807e-bb964f55e505 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:55.555806 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e4a0df-d0fe-4f75-807e-bb964f55e505 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:55.556078 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e4a0df-d0fe-4f75-807e-bb964f55e505 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:55.556851 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e4a0df-d0fe-4f75-807e-bb964f55e505 None admin] RBAC: Adding query filter params (role.id=62d96ac9513c452894fc67b442cfc59f, scope.project.id=fbb3348b420b4f6ba5d171c387e5a403, user.id=91c065b6fe104421a64d9eb474a7761c) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:27:55.557214 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e4a0df-d0fe-4f75-807e-bb964f55e505 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:55.559610 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e4a0df-d0fe-4f75-807e-bb964f55e505 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:55.567065 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 103/199] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 15:27:55 2022] GET /identity/v3/role_assignments?user.id=91c065b6fe104421a64d9eb474a7761c&scope.project.id=fbb3348b420b4f6ba5d171c387e5a403&role.id=62d96ac9513c452894fc67b442cfc59f => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:55.574424 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723ae8d-3c3e-4632-8dfe-415f9cf66941 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:55.574746 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723ae8d-3c3e-4632-8dfe-415f9cf66941 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:55.574978 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723ae8d-3c3e-4632-8dfe-415f9cf66941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:55.611007 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-c723ae8d-3c3e-4632-8dfe-415f9cf66941 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:27:55.612638 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c723ae8d-3c3e-4632-8dfe-415f9cf66941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:27:55.648735 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5109a33-fe1b-4184-bfb7-f33a0278b693 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:55.698297 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 104/200] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:27:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:27:55.740301 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffceaf98-2387-48a6-88b6-a67aeaf38acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:55.740877 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffceaf98-2387-48a6-88b6-a67aeaf38acc None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:55.741152 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffceaf98-2387-48a6-88b6-a67aeaf38acc None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:55.741418 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffceaf98-2387-48a6-88b6-a67aeaf38acc None admin] PATH_INFO: `/v3/projects/fbb3348b420b4f6ba5d171c387e5a403` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:55.746744 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffceaf98-2387-48a6-88b6-a67aeaf38acc None admin] RBAC: Authorizing `identity:get_project(project_id=fbb3348b420b4f6ba5d171c387e5a403)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:55.748223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffceaf98-2387-48a6-88b6-a67aeaf38acc None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:55.749990 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 97/201] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:27:55 2022] GET /identity/v3/projects/fbb3348b420b4f6ba5d171c387e5a403 => generated 302 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:55.756769 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c723ae8d-3c3e-4632-8dfe-415f9cf66941 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:55.862674 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:55.878275 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c723ae8d-3c3e-4632-8dfe-415f9cf66941 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:55.922947 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124287e6-9ff8-4f67-853d-5188e0d8a57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:55.923573 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124287e6-9ff8-4f67-853d-5188e0d8a57a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:55.923851 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124287e6-9ff8-4f67-853d-5188e0d8a57a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:55.924121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124287e6-9ff8-4f67-853d-5188e0d8a57a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:55.924849 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124287e6-9ff8-4f67-853d-5188e0d8a57a None admin] RBAC: Adding query filter params (role.id=5038c8140b7145ceb7494210a5c9f9f0, scope.project.id=fbb3348b420b4f6ba5d171c387e5a403, user.id=91c065b6fe104421a64d9eb474a7761c) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:27:55.925153 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124287e6-9ff8-4f67-853d-5188e0d8a57a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:55.926616 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124287e6-9ff8-4f67-853d-5188e0d8a57a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:55.933512 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 105/202] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 15:27:55 2022] GET /identity/v3/role_assignments?user.id=91c065b6fe104421a64d9eb474a7761c&scope.project.id=fbb3348b420b4f6ba5d171c387e5a403&role.id=5038c8140b7145ceb7494210a5c9f9f0 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:56.041691 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffceaf98-2387-48a6-88b6-a67aeaf38acc None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:56.096195 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:56.157547 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffceaf98-2387-48a6-88b6-a67aeaf38acc None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:56.195289 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77511ef2-3d73-4402-b43e-b60408bf1577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:56.196327 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77511ef2-3d73-4402-b43e-b60408bf1577 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:56.196805 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77511ef2-3d73-4402-b43e-b60408bf1577 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:56.197272 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77511ef2-3d73-4402-b43e-b60408bf1577 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:56.209494 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77511ef2-3d73-4402-b43e-b60408bf1577 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:27:56.215577 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-77511ef2-3d73-4402-b43e-b60408bf1577 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 15:27:56.215577 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:27:56.215577 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:27:56.215577 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:27:56.215577 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:27:56.215577 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:27:56.215577 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:27:56.215577 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:27:56.215577 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:27:56.215577 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:27:56.215577 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:27:56.215577 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:27:56.215577 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:27:56.215577 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:27:56.215577 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:27:56.215577 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:27:56.216576 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:27:56.216576 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:27:56.216576 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:27:56.216576 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:27:56.216576 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:27:56.216576 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:27:56.216576 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:27:56.216576 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:27:56.216576 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:27:56.216576 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:27:56.216576 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:27:56.216576 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:27:56.216576 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:27:56.216576 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:27:56.216576 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:27:56.216576 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:27:56.217364 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:27:56.217364 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:27:56.217364 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:27:56.217364 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:27:56.217364 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:27:56.217364 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:27:56.217364 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:27:56.217364 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:27:56.217364 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:27:56.217364 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:27:56.217364 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 15:27:56.217364 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:27:56.225903 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 98/203] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:27:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:27:56.231128 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124287e6-9ff8-4f67-853d-5188e0d8a57a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:56.388911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124287e6-9ff8-4f67-853d-5188e0d8a57a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:56.440251 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65bbd88-3c15-4b59-b320-c7d2b978fa0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:56.441207 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65bbd88-3c15-4b59-b320-c7d2b978fa0a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:56.441690 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65bbd88-3c15-4b59-b320-c7d2b978fa0a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:56.442185 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65bbd88-3c15-4b59-b320-c7d2b978fa0a None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:56.443043 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65bbd88-3c15-4b59-b320-c7d2b978fa0a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:27:56.443569 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65bbd88-3c15-4b59-b320-c7d2b978fa0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:56.445145 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65bbd88-3c15-4b59-b320-c7d2b978fa0a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:56.457116 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 106/204] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 15:27:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:56.463264 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77511ef2-3d73-4402-b43e-b60408bf1577 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:56.586966 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77511ef2-3d73-4402-b43e-b60408bf1577 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:56.599480 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:56.633603 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3258600c-4dce-4d9d-9d23-5a66f9d287bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:56.634269 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3258600c-4dce-4d9d-9d23-5a66f9d287bd None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:56.634551 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3258600c-4dce-4d9d-9d23-5a66f9d287bd None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:56.634822 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3258600c-4dce-4d9d-9d23-5a66f9d287bd None admin] PATH_INFO: `/v3/users/9f3930cf37644c92bae454a9ddcc1670` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:56.636496 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3258600c-4dce-4d9d-9d23-5a66f9d287bd None admin] RBAC: Authorizing `identity:get_user(user_id=9f3930cf37644c92bae454a9ddcc1670)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:56.637950 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3258600c-4dce-4d9d-9d23-5a66f9d287bd None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:56.639524 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 99/205] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:27:56 2022] GET /identity/v3/users/9f3930cf37644c92bae454a9ddcc1670 => generated 289 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:56.645022 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65bbd88-3c15-4b59-b320-c7d2b978fa0a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:56.668961 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:56.772055 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65bbd88-3c15-4b59-b320-c7d2b978fa0a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:56.815193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83e04fd-7031-4a3c-9e08-e6786a780924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:56.815863 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e04fd-7031-4a3c-9e08-e6786a780924 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:56.816151 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e04fd-7031-4a3c-9e08-e6786a780924 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:56.816433 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e04fd-7031-4a3c-9e08-e6786a780924 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:56.905203 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83e04fd-7031-4a3c-9e08-e6786a780924 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:56.907030 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83e04fd-7031-4a3c-9e08-e6786a780924 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:56.931393 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:57.041193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 107/206] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:27:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:27:57.047342 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3258600c-4dce-4d9d-9d23-5a66f9d287bd None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:57.156442 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3258600c-4dce-4d9d-9d23-5a66f9d287bd None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:57.191344 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a657ec-6486-4ff5-8334-922d276f6c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:57.191976 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a657ec-6486-4ff5-8334-922d276f6c9c None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:57.192254 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a657ec-6486-4ff5-8334-922d276f6c9c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:57.192526 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a657ec-6486-4ff5-8334-922d276f6c9c None admin] PATH_INFO: `/v3/projects/4251d157536c4934bf6f66268f3037a9` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:57.193664 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a657ec-6486-4ff5-8334-922d276f6c9c None admin] RBAC: Authorizing `identity:get_project(project_id=4251d157536c4934bf6f66268f3037a9)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:57.195142 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a657ec-6486-4ff5-8334-922d276f6c9c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:57.196889 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 100/207] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:27:57 2022] GET /identity/v3/projects/4251d157536c4934bf6f66268f3037a9 => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:57.202820 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83e04fd-7031-4a3c-9e08-e6786a780924 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:57.349487 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83e04fd-7031-4a3c-9e08-e6786a780924 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:57.372733 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:57.390457 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caddf99-20c4-4e34-ab78-87b00d909689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:57.391060 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caddf99-20c4-4e34-ab78-87b00d909689 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:57.391337 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caddf99-20c4-4e34-ab78-87b00d909689 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:57.391604 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caddf99-20c4-4e34-ab78-87b00d909689 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:57.392316 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caddf99-20c4-4e34-ab78-87b00d909689 None admin] RBAC: Adding query filter params (role.id=9e4cabcdb1e349c699da538ce00d1396, scope.project.id=4251d157536c4934bf6f66268f3037a9, user.id=9f3930cf37644c92bae454a9ddcc1670) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:27:57.392618 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caddf99-20c4-4e34-ab78-87b00d909689 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:57.394090 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caddf99-20c4-4e34-ab78-87b00d909689 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:57.399915 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 108/208] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 15:27:57 2022] GET /identity/v3/role_assignments?user.id=9f3930cf37644c92bae454a9ddcc1670&scope.project.id=4251d157536c4934bf6f66268f3037a9&role.id=9e4cabcdb1e349c699da538ce00d1396 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:57.828310 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714ba49-664f-46f3-b418-716946c4a95a None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:57.828864 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714ba49-664f-46f3-b418-716946c4a95a None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:57.828864 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714ba49-664f-46f3-b418-716946c4a95a None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:57.830502 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 101/209] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:27:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:27:57.836792 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c69950c-17ef-4c80-a809-62a1f2c9a501 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:57.837274 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c69950c-17ef-4c80-a809-62a1f2c9a501 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:57.837343 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c69950c-17ef-4c80-a809-62a1f2c9a501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:57.855652 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-6c69950c-17ef-4c80-a809-62a1f2c9a501 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:27:57.856846 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6c69950c-17ef-4c80-a809-62a1f2c9a501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:27:57.933896 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 109/210] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:27:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:27:57.942913 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd1658d-cad4-4c4e-aee2-028b7ba896db None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:57.943440 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd1658d-cad4-4c4e-aee2-028b7ba896db None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:57.943923 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd1658d-cad4-4c4e-aee2-028b7ba896db None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:57.945434 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 102/211] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:27:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:27:57.950834 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db7360-7d21-4122-a8c7-52bad3b72000 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:57.951123 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db7360-7d21-4122-a8c7-52bad3b72000 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:57.951402 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db7360-7d21-4122-a8c7-52bad3b72000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:57.974108 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-81db7360-7d21-4122-a8c7-52bad3b72000 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:27:57.975352 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-81db7360-7d21-4122-a8c7-52bad3b72000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:27:58.054942 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 110/212] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:27:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:27:58.231656 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd1658d-cad4-4c4e-aee2-028b7ba896db None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:58.257746 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e9c9b-e01e-4343-8754-cabbbffa6c88 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:58.258098 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e9c9b-e01e-4343-8754-cabbbffa6c88 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:58.258335 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e9c9b-e01e-4343-8754-cabbbffa6c88 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:58.260072 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 111/213] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:27:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:27:58.265832 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea17f99-351c-464a-9936-37d8050e1659 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:58.266147 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea17f99-351c-464a-9936-37d8050e1659 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:58.266421 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea17f99-351c-464a-9936-37d8050e1659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:58.284717 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:58.287783 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-dea17f99-351c-464a-9936-37d8050e1659 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:27:58.288976 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-dea17f99-351c-464a-9936-37d8050e1659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:27:58.328030 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:58.353818 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd1658d-cad4-4c4e-aee2-028b7ba896db None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:58.364090 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 112/214] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:27:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:27:58.378610 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea17f99-351c-464a-9936-37d8050e1659 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:58.400070 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cbd357-faf6-4067-8d91-5270d9682af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:58.400785 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cbd357-faf6-4067-8d91-5270d9682af4 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:58.401067 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cbd357-faf6-4067-8d91-5270d9682af4 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:58.401978 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cbd357-faf6-4067-8d91-5270d9682af4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:58.403475 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cbd357-faf6-4067-8d91-5270d9682af4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:58.405430 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cbd357-faf6-4067-8d91-5270d9682af4 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:58.411789 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 103/215] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:27:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:58.412950 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cbd357-faf6-4067-8d91-5270d9682af4 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:58.505183 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea17f99-351c-464a-9936-37d8050e1659 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:58.518118 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cbd357-faf6-4067-8d91-5270d9682af4 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:58.530460 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:58.547744 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fa1929-7d6c-44a3-90ac-a44a44ac642b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:58.548388 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa1929-7d6c-44a3-90ac-a44a44ac642b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:58.548675 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa1929-7d6c-44a3-90ac-a44a44ac642b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:58.548949 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa1929-7d6c-44a3-90ac-a44a44ac642b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:58.558277 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c34b2-0483-42da-9f54-2b41cb48e69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:58.558969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fa1929-7d6c-44a3-90ac-a44a44ac642b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:27:58.559321 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c34b2-0483-42da-9f54-2b41cb48e69d None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:58.559748 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c34b2-0483-42da-9f54-2b41cb48e69d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:58.560156 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c34b2-0483-42da-9f54-2b41cb48e69d None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:58.560949 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5c34b2-0483-42da-9f54-2b41cb48e69d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:58.562530 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5c34b2-0483-42da-9f54-2b41cb48e69d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:58.563802 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-82fa1929-7d6c-44a3-90ac-a44a44ac642b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:27:58.563802 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:27:58.563802 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:27:58.563802 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:27:58.563802 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:27:58.563802 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:27:58.563802 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:27:58.563802 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:27:58.563802 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:27:58.563802 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:27:58.563802 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:27:58.563802 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:27:58.563802 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:27:58.563802 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:27:58.563802 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:27:58.563802 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:27:58.564865 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:27:58.564865 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:27:58.564865 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:27:58.564865 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:27:58.564865 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:27:58.564865 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:27:58.564865 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:27:58.564865 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:27:58.564865 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:27:58.564865 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:27:58.564865 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:27:58.564865 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:27:58.564865 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:27:58.564865 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:27:58.564865 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:27:58.564865 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:27:58.570657 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:27:58.570657 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:27:58.570657 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:27:58.570657 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:27:58.570657 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:27:58.570657 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:27:58.570657 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:27:58.570657 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:27:58.570657 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:27:58.570657 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:27:58.570657 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:27:58.570657 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:27:58.570657 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 113/216] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:27:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:27:58.570657 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868e8cf-d023-4447-af44-13b266b9b015 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:58.571339 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868e8cf-d023-4447-af44-13b266b9b015 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:58.571339 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868e8cf-d023-4447-af44-13b266b9b015 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:58.571339 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 114/217] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:27:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:27:58.571339 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7868e8cf-d023-4447-af44-13b266b9b015 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:58.599009 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 104/218] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 15:27:58 2022] POST /identity/v3/users => generated 297 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:27:58.601134 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6535fa9-5ec4-4a43-9085-db8ef25aeae0 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:58.601680 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6535fa9-5ec4-4a43-9085-db8ef25aeae0 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:58.602224 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6535fa9-5ec4-4a43-9085-db8ef25aeae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:58.608640 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:58.628895 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-b6535fa9-5ec4-4a43-9085-db8ef25aeae0 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:27:58.630903 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b6535fa9-5ec4-4a43-9085-db8ef25aeae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:27:58.686749 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7868e8cf-d023-4447-af44-13b266b9b015 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:58.711346 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 105/219] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:27:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:27:58.712083 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6535fa9-5ec4-4a43-9085-db8ef25aeae0 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:58.748530 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a11de-74e8-444d-8dc6-cb844641e63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:58.750316 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a11de-74e8-444d-8dc6-cb844641e63c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:58.750902 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a11de-74e8-444d-8dc6-cb844641e63c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:58.751353 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a11de-74e8-444d-8dc6-cb844641e63c None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:58.752542 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8a11de-74e8-444d-8dc6-cb844641e63c None admin] RBAC: Adding query filter params (name=member) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:27:58.753032 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8a11de-74e8-444d-8dc6-cb844641e63c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:58.755469 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8a11de-74e8-444d-8dc6-cb844641e63c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:58.768562 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 115/220] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 15:27:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:27:58.774970 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a11de-74e8-444d-8dc6-cb844641e63c None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:58.840748 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:58.852079 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6535fa9-5ec4-4a43-9085-db8ef25aeae0 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:58.896359 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846a6fe2-0d96-4bdb-980a-c974c5703f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:58.897339 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a6fe2-0d96-4bdb-980a-c974c5703f6e None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:58.897339 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a6fe2-0d96-4bdb-980a-c974c5703f6e None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:58.897523 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846a6fe2-0d96-4bdb-980a-c974c5703f6e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:58.900813 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:58.910826 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a11de-74e8-444d-8dc6-cb844641e63c None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:58.917166 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846a6fe2-0d96-4bdb-980a-c974c5703f6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:27:58.923605 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-846a6fe2-0d96-4bdb-980a-c974c5703f6e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:27:58.923605 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:27:58.923605 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:27:58.923605 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:27:58.923605 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:27:58.923605 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:27:58.923605 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:27:58.923605 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:27:58.923605 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:27:58.923605 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:27:58.923605 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:27:58.923605 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:27:58.923605 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:27:58.923605 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:27:58.923605 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:27:58.923605 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:27:58.924499 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:27:58.924499 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:27:58.924499 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:27:58.924499 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:27:58.924499 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:27:58.924499 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:27:58.924499 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:27:58.924499 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:27:58.924499 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:27:58.924499 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:27:58.924499 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:27:58.924499 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:27:58.924499 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:27:58.924499 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:27:58.924499 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:27:58.924499 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:27:58.925270 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:27:58.925270 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:27:58.925270 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:27:58.925270 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:27:58.925270 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:27:58.925270 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:27:58.925270 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:27:58.925270 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:27:58.925270 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:27:58.925270 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:27:58.925270 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:27:58.925270 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:27:58.926345 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 106/221] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 15:27:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:27:58.930948 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846a6fe2-0d96-4bdb-980a-c974c5703f6e None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:58.959987 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d54e65-e31b-485e-9bee-677671983374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:58.960672 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d54e65-e31b-485e-9bee-677671983374 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:58.961021 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d54e65-e31b-485e-9bee-677671983374 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:58.961326 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d54e65-e31b-485e-9bee-677671983374 None admin] PATH_INFO: `/v3/users/0b8b86dee823406ba22c5057462816c0` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:58.980240 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d54e65-e31b-485e-9bee-677671983374 None admin] RBAC: Authorizing `identity:get_user(user_id=0b8b86dee823406ba22c5057462816c0)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:58.981765 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d54e65-e31b-485e-9bee-677671983374 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:58.983640 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 116/222] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:27:58 2022] GET /identity/v3/users/0b8b86dee823406ba22c5057462816c0 => generated 297 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:58.994050 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d54e65-e31b-485e-9bee-677671983374 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:59.062884 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846a6fe2-0d96-4bdb-980a-c974c5703f6e None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:59.111093 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc1d75-2926-4739-9696-1023a8d1a31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:59.111835 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc1d75-2926-4739-9696-1023a8d1a31b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:59.112146 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc1d75-2926-4739-9696-1023a8d1a31b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:59.112525 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc1d75-2926-4739-9696-1023a8d1a31b None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:59.113198 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cc1d75-2926-4739-9696-1023a8d1a31b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:27:59.113503 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cc1d75-2926-4739-9696-1023a8d1a31b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:59.115631 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cc1d75-2926-4739-9696-1023a8d1a31b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:59.117881 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d54e65-e31b-485e-9bee-677671983374 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:59.126974 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 107/223] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:27:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:59.127948 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc1d75-2926-4739-9696-1023a8d1a31b None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:59.160369 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ace680b-942a-4135-8735-578232d2cbbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:59.161265 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace680b-942a-4135-8735-578232d2cbbd None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:59.161899 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace680b-942a-4135-8735-578232d2cbbd None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:59.162814 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace680b-942a-4135-8735-578232d2cbbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:59.166208 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:59.192345 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:59.247788 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc1d75-2926-4739-9696-1023a8d1a31b None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:59.264855 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ace680b-942a-4135-8735-578232d2cbbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:59.266377 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ace680b-942a-4135-8735-578232d2cbbd None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:59.299607 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89765d92-c730-444c-b147-e65960adb9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:59.300494 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89765d92-c730-444c-b147-e65960adb9d0 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:59.300991 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89765d92-c730-444c-b147-e65960adb9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:59.301431 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89765d92-c730-444c-b147-e65960adb9d0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:59.313216 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89765d92-c730-444c-b147-e65960adb9d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:27:59.318610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-89765d92-c730-444c-b147-e65960adb9d0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:27:59.318610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:27:59.318610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:27:59.318610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:27:59.318610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:27:59.318610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:27:59.318610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:27:59.318610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:27:59.318610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:27:59.318610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:27:59.318610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:27:59.318610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:27:59.318610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:27:59.318610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:27:59.318610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:27:59.318610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:27:59.319482 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:27:59.319482 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:27:59.319482 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:27:59.319482 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:27:59.319482 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:27:59.319482 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:27:59.319482 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:27:59.319482 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:27:59.319482 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:27:59.319482 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:27:59.319482 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:27:59.319482 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:27:59.319482 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:27:59.319482 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:27:59.319482 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:27:59.319482 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:27:59.320268 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:27:59.320268 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:27:59.320268 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:27:59.320268 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:27:59.320268 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:27:59.320268 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:27:59.320268 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:27:59.320268 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:27:59.320268 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:27:59.320268 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:27:59.320268 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:27:59.320268 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:27:59.322974 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 108/224] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 15:27:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:27:59.324189 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89765d92-c730-444c-b147-e65960adb9d0 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:59.388099 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 117/225] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:27:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:27:59.389279 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ace680b-942a-4135-8735-578232d2cbbd None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:59.418231 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:59.446850 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89765d92-c730-444c-b147-e65960adb9d0 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:59.464566 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:59.489097 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c21bc1-0c00-4a27-b211-3e4df85dbfd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:59.489878 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c21bc1-0c00-4a27-b211-3e4df85dbfd4 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:59.490228 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c21bc1-0c00-4a27-b211-3e4df85dbfd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:59.490564 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c21bc1-0c00-4a27-b211-3e4df85dbfd4 None admin] PATH_INFO: `/v3/users/91c065b6fe104421a64d9eb474a7761c` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:59.492225 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c21bc1-0c00-4a27-b211-3e4df85dbfd4 None admin] RBAC: Authorizing `identity:get_user(user_id=91c065b6fe104421a64d9eb474a7761c)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:59.494245 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c21bc1-0c00-4a27-b211-3e4df85dbfd4 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:59.497240 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 109/226] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:27:59 2022] GET /identity/v3/users/91c065b6fe104421a64d9eb474a7761c => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:59.502266 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c21bc1-0c00-4a27-b211-3e4df85dbfd4 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:59.519294 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ace680b-942a-4135-8735-578232d2cbbd None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:59.556363 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0664d2-4eb3-4ee7-9431-c3270e6537c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:59.556977 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0664d2-4eb3-4ee7-9431-c3270e6537c7 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:59.557255 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0664d2-4eb3-4ee7-9431-c3270e6537c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:59.558072 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0664d2-4eb3-4ee7-9431-c3270e6537c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:59.558771 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0664d2-4eb3-4ee7-9431-c3270e6537c7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:27:59.559075 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0664d2-4eb3-4ee7-9431-c3270e6537c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:59.560679 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0664d2-4eb3-4ee7-9431-c3270e6537c7 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:59.569737 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 118/227] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:27:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:59.575616 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0664d2-4eb3-4ee7-9431-c3270e6537c7 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:59.605882 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c21bc1-0c00-4a27-b211-3e4df85dbfd4 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:59.641337 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8517a9-7bfd-477a-b935-d68661a1f0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:59.641998 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8517a9-7bfd-477a-b935-d68661a1f0bb None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:59.642272 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8517a9-7bfd-477a-b935-d68661a1f0bb None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:59.642536 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8517a9-7bfd-477a-b935-d68661a1f0bb None admin] PATH_INFO: `/v3/projects/fbb3348b420b4f6ba5d171c387e5a403` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:59.643717 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8517a9-7bfd-477a-b935-d68661a1f0bb None admin] RBAC: Authorizing `identity:get_project(project_id=fbb3348b420b4f6ba5d171c387e5a403)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:59.645160 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8517a9-7bfd-477a-b935-d68661a1f0bb None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:59.646965 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 110/228] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:27:59 2022] GET /identity/v3/projects/fbb3348b420b4f6ba5d171c387e5a403 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:59.647554 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8517a9-7bfd-477a-b935-d68661a1f0bb None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:59.663969 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:59.689898 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0664d2-4eb3-4ee7-9431-c3270e6537c7 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:59.705309 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:59.728765 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bf3fdc-1ec8-443d-9d07-1ff33b258ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:59.729439 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf3fdc-1ec8-443d-9d07-1ff33b258ee0 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:59.729721 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf3fdc-1ec8-443d-9d07-1ff33b258ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:59.730024 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf3fdc-1ec8-443d-9d07-1ff33b258ee0 None admin] PATH_INFO: `/v3/projects/fbb3348b420b4f6ba5d171c387e5a403` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:59.731271 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bf3fdc-1ec8-443d-9d07-1ff33b258ee0 None admin] RBAC: Authorizing `identity:get_project(project_id=fbb3348b420b4f6ba5d171c387e5a403)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:59.733068 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bf3fdc-1ec8-443d-9d07-1ff33b258ee0 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:59.735169 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 119/229] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:27:59 2022] GET /identity/v3/projects/fbb3348b420b4f6ba5d171c387e5a403 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:59.735918 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bf3fdc-1ec8-443d-9d07-1ff33b258ee0 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:59.759990 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8517a9-7bfd-477a-b935-d68661a1f0bb None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:59.798342 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b3aec6-8de7-4882-8cee-0c295903ce68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:59.798972 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3aec6-8de7-4882-8cee-0c295903ce68 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:59.799243 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3aec6-8de7-4882-8cee-0c295903ce68 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:59.799509 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3aec6-8de7-4882-8cee-0c295903ce68 None admin] PATH_INFO: `/v3/users/91c065b6fe104421a64d9eb474a7761c` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:59.800627 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b3aec6-8de7-4882-8cee-0c295903ce68 None admin] RBAC: Authorizing `identity:get_user(user_id=91c065b6fe104421a64d9eb474a7761c)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:59.802098 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b3aec6-8de7-4882-8cee-0c295903ce68 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:59.803946 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 111/230] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:27:59 2022] GET /identity/v3/users/91c065b6fe104421a64d9eb474a7761c => generated 283 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:59.804737 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b3aec6-8de7-4882-8cee-0c295903ce68 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:59.854023 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bf3fdc-1ec8-443d-9d07-1ff33b258ee0 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:59.889833 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:59.891110 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92023c7a-2110-49d6-8e79-9f6b5531599e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:59.891809 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92023c7a-2110-49d6-8e79-9f6b5531599e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:59.892089 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92023c7a-2110-49d6-8e79-9f6b5531599e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:59.892757 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92023c7a-2110-49d6-8e79-9f6b5531599e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:59.893129 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92023c7a-2110-49d6-8e79-9f6b5531599e None admin] RBAC: Adding query filter params (role.id=e152a36b5a1b470b912b1d705462f512, scope.project.id=fbb3348b420b4f6ba5d171c387e5a403, user.id=0b8b86dee823406ba22c5057462816c0) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:27:59.893439 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92023c7a-2110-49d6-8e79-9f6b5531599e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:59.894907 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92023c7a-2110-49d6-8e79-9f6b5531599e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:59.900983 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 120/231] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 15:27:59 2022] GET /identity/v3/role_assignments?user.id=0b8b86dee823406ba22c5057462816c0&scope.project.id=fbb3348b420b4f6ba5d171c387e5a403&role.id=e152a36b5a1b470b912b1d705462f512 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:27:59.903641 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92023c7a-2110-49d6-8e79-9f6b5531599e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:27:59.909169 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b3aec6-8de7-4882-8cee-0c295903ce68 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:27:59.946168 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a9bf8-c633-4e86-a99c-d7bd9e4275ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:27:59.946835 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a9bf8-c633-4e86-a99c-d7bd9e4275ed None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:27:59.947108 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a9bf8-c633-4e86-a99c-d7bd9e4275ed None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:27:59.947378 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a9bf8-c633-4e86-a99c-d7bd9e4275ed None admin] PATH_INFO: `/v3/projects/fbb3348b420b4f6ba5d171c387e5a403/users/91c065b6fe104421a64d9eb474a7761c/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:27:59.949377 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6a9bf8-c633-4e86-a99c-d7bd9e4275ed None admin] RBAC: Authorizing `identity:create_grant(project_id=fbb3348b420b4f6ba5d171c387e5a403, user_id=91c065b6fe104421a64d9eb474a7761c, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:27:59.950876 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6a9bf8-c633-4e86-a99c-d7bd9e4275ed None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:27:59.956825 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:27:59.974129 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 112/232] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 15:27:59 2022] PUT /identity/v3/projects/fbb3348b420b4f6ba5d171c387e5a403/users/91c065b6fe104421a64d9eb474a7761c/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:28:00.019195 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92023c7a-2110-49d6-8e79-9f6b5531599e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:00.058440 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e288bf-b6c3-4bcf-a015-0d47c6a5dd24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:00.059100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e288bf-b6c3-4bcf-a015-0d47c6a5dd24 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:00.059379 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e288bf-b6c3-4bcf-a015-0d47c6a5dd24 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:00.059651 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e288bf-b6c3-4bcf-a015-0d47c6a5dd24 None admin] PATH_INFO: `/v3/projects/fbb3348b420b4f6ba5d171c387e5a403` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:00.060900 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e288bf-b6c3-4bcf-a015-0d47c6a5dd24 None admin] RBAC: Authorizing `identity:get_project(project_id=fbb3348b420b4f6ba5d171c387e5a403)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:00.062412 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e288bf-b6c3-4bcf-a015-0d47c6a5dd24 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:00.069068 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 121/233] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:27:59 2022] GET /identity/v3/projects/fbb3348b420b4f6ba5d171c387e5a403 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:00.075029 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a9bf8-c633-4e86-a99c-d7bd9e4275ed None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:00.207368 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a9bf8-c633-4e86-a99c-d7bd9e4275ed None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:00.244490 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73188fdd-b9fa-4e46-a67a-0282b14a8480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:00.246095 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73188fdd-b9fa-4e46-a67a-0282b14a8480 None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:00.246564 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73188fdd-b9fa-4e46-a67a-0282b14a8480 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:00.247021 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73188fdd-b9fa-4e46-a67a-0282b14a8480 None admin] PATH_INFO: `/v3/projects/fbb3348b420b4f6ba5d171c387e5a403/users/91c065b6fe104421a64d9eb474a7761c/roles/5038c8140b7145ceb7494210a5c9f9f0` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:00.249451 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73188fdd-b9fa-4e46-a67a-0282b14a8480 None admin] RBAC: Authorizing `identity:create_grant(project_id=fbb3348b420b4f6ba5d171c387e5a403, user_id=91c065b6fe104421a64d9eb474a7761c, role_id=5038c8140b7145ceb7494210a5c9f9f0)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:00.251188 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73188fdd-b9fa-4e46-a67a-0282b14a8480 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:00.267836 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 113/234] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 15:28:00 2022] PUT /identity/v3/projects/fbb3348b420b4f6ba5d171c387e5a403/users/91c065b6fe104421a64d9eb474a7761c/roles/5038c8140b7145ceb7494210a5c9f9f0 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:28:01.358130 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23e9e7-5212-4a7a-9ea7-4a736407589d None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:01.358688 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23e9e7-5212-4a7a-9ea7-4a736407589d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:01.358688 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23e9e7-5212-4a7a-9ea7-4a736407589d None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:01.360321 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 122/235] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:01.366129 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3c6cc-1ea1-41dd-84ba-9dd9f9b496fd None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:01.366604 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3c6cc-1ea1-41dd-84ba-9dd9f9b496fd None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:01.367080 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3c6cc-1ea1-41dd-84ba-9dd9f9b496fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:01.382068 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:01.399153 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-16d3c6cc-1ea1-41dd-84ba-9dd9f9b496fd None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:01.400357 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-16d3c6cc-1ea1-41dd-84ba-9dd9f9b496fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:01.483768 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da8ff03-a366-48ff-87ac-aedbc9903e5d None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:01.484095 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da8ff03-a366-48ff-87ac-aedbc9903e5d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:01.484317 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da8ff03-a366-48ff-87ac-aedbc9903e5d None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:01.485648 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 123/236] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:28:01.497422 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fc703d-626a-4cb3-ae65-b1b031164486 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:01.497700 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fc703d-626a-4cb3-ae65-b1b031164486 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:01.497998 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fc703d-626a-4cb3-ae65-b1b031164486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:01.512932 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 114/237] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:28:01.527342 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-c6fc703d-626a-4cb3-ae65-b1b031164486 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:01.528539 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c6fc703d-626a-4cb3-ae65-b1b031164486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:01.624778 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 124/238] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:28:01.797909 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3c6cc-1ea1-41dd-84ba-9dd9f9b496fd None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:01.903486 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3c6cc-1ea1-41dd-84ba-9dd9f9b496fd None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:01.938209 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:01.945302 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda3514-93c1-471d-a44d-13724c35f0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:01.945970 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda3514-93c1-471d-a44d-13724c35f0aa None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:01.946248 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda3514-93c1-471d-a44d-13724c35f0aa None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:01.946519 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda3514-93c1-471d-a44d-13724c35f0aa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:01.963023 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda3514-93c1-471d-a44d-13724c35f0aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:01.968047 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-1eda3514-93c1-471d-a44d-13724c35f0aa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 15:28:01.968047 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:01.968047 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:01.968047 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:01.968047 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:01.968047 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:01.968047 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:01.968047 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:01.968047 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:01.968047 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:01.968047 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:01.968047 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:01.968047 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:01.968047 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:01.968047 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:01.968047 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:01.969192 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:01.969192 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:01.969192 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:01.969192 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:01.969192 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:01.969192 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:01.969192 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:01.969192 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:01.969192 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:01.969192 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:01.969192 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:01.969192 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:01.969192 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:01.969192 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:01.969192 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:01.969192 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:01.970465 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:01.970465 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:01.970465 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:01.970465 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:01.970465 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:01.970465 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:01.970465 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:01.970465 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:01.970465 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:01.970465 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:01.970465 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 15:28:01.970465 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:01.973910 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 115/239] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:28:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:01.974934 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fc703d-626a-4cb3-ae65-b1b031164486 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:02.035697 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda3514-93c1-471d-a44d-13724c35f0aa None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:02.070182 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:02.129364 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fc703d-626a-4cb3-ae65-b1b031164486 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:02.165785 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f77cef-e483-4a9f-a9c3-6ea8852f901e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:02.166431 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f77cef-e483-4a9f-a9c3-6ea8852f901e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:02.166850 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f77cef-e483-4a9f-a9c3-6ea8852f901e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:02.167132 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f77cef-e483-4a9f-a9c3-6ea8852f901e None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:02.167895 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f77cef-e483-4a9f-a9c3-6ea8852f901e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:02.168201 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f77cef-e483-4a9f-a9c3-6ea8852f901e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:02.169669 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f77cef-e483-4a9f-a9c3-6ea8852f901e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:02.178605 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 125/240] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 15:28:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:02.183915 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f77cef-e483-4a9f-a9c3-6ea8852f901e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:02.209647 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda3514-93c1-471d-a44d-13724c35f0aa None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:02.285355 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2226d25-5e1d-45eb-92e2-7a2b1319b558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:02.286067 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2226d25-5e1d-45eb-92e2-7a2b1319b558 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:02.286344 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2226d25-5e1d-45eb-92e2-7a2b1319b558 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:02.286646 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2226d25-5e1d-45eb-92e2-7a2b1319b558 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:02.287767 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2226d25-5e1d-45eb-92e2-7a2b1319b558 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:02.289202 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2226d25-5e1d-45eb-92e2-7a2b1319b558 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:02.290950 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 116/241] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:28:02.310096 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2226d25-5e1d-45eb-92e2-7a2b1319b558 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:02.374668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f77cef-e483-4a9f-a9c3-6ea8852f901e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:02.398209 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:02.425690 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:02.443823 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41912da9-9ae7-427d-988f-46f8562de219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:02.444695 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41912da9-9ae7-427d-988f-46f8562de219 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:02.444971 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41912da9-9ae7-427d-988f-46f8562de219 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:02.446257 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41912da9-9ae7-427d-988f-46f8562de219 None admin] PATH_INFO: `/v3/users/6c933b0818414e45a45823374f8e61e1` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:02.473840 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41912da9-9ae7-427d-988f-46f8562de219 None admin] RBAC: Authorizing `identity:get_user(user_id=6c933b0818414e45a45823374f8e61e1)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:02.475585 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41912da9-9ae7-427d-988f-46f8562de219 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:02.478737 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 126/242] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:28:02 2022] GET /identity/v3/users/6c933b0818414e45a45823374f8e61e1 => generated 297 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:02.483022 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41912da9-9ae7-427d-988f-46f8562de219 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:02.501027 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2226d25-5e1d-45eb-92e2-7a2b1319b558 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:02.565093 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ca3906-66ea-4cb0-b468-387b1d4a756e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:02.565753 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca3906-66ea-4cb0-b468-387b1d4a756e None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:02.566059 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca3906-66ea-4cb0-b468-387b1d4a756e None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:02.566333 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca3906-66ea-4cb0-b468-387b1d4a756e None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:02.566992 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ca3906-66ea-4cb0-b468-387b1d4a756e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:02.568438 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ca3906-66ea-4cb0-b468-387b1d4a756e None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:02.598872 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 117/243] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 15:28:02 2022] POST /identity/v3/users => generated 293 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:28:02.600289 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd562c-bed2-414d-8eaf-725baa8b3698 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:02.600620 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd562c-bed2-414d-8eaf-725baa8b3698 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:02.601057 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd562c-bed2-414d-8eaf-725baa8b3698 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:02.607216 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 118/244] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:02.608649 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7017f682-0b04-4bd7-bb5f-612b57ff8543 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:02.608938 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7017f682-0b04-4bd7-bb5f-612b57ff8543 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:02.609252 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7017f682-0b04-4bd7-bb5f-612b57ff8543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:02.630106 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-7017f682-0b04-4bd7-bb5f-612b57ff8543 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:02.631884 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7017f682-0b04-4bd7-bb5f-612b57ff8543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:02.638530 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41912da9-9ae7-427d-988f-46f8562de219 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:02.678343 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9979b82-255f-4b92-af03-67e39d168a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:02.678886 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9979b82-255f-4b92-af03-67e39d168a12 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:02.679164 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9979b82-255f-4b92-af03-67e39d168a12 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:02.679434 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9979b82-255f-4b92-af03-67e39d168a12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:02.709536 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 119/245] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:02.715534 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:02.755658 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9979b82-255f-4b92-af03-67e39d168a12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:02.757413 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9979b82-255f-4b92-af03-67e39d168a12 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:02.871601 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 127/246] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:28:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:28:02.877027 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7017f682-0b04-4bd7-bb5f-612b57ff8543 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:02.943947 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:02.970726 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f230aee8-00a1-4b56-aa47-99d447cc02cb None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:02.971343 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f230aee8-00a1-4b56-aa47-99d447cc02cb None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:02.972101 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f230aee8-00a1-4b56-aa47-99d447cc02cb None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:02.975092 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 128/247] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:28:02.976464 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e0fce-1247-42a9-af27-56912c97a7e2 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:02.977020 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e0fce-1247-42a9-af27-56912c97a7e2 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:02.977451 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e0fce-1247-42a9-af27-56912c97a7e2 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:02.981176 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 129/248] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:02.982774 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e91f6-0991-4792-a25c-5519868576f9 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:02.983252 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e91f6-0991-4792-a25c-5519868576f9 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:02.983735 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e91f6-0991-4792-a25c-5519868576f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:03.002370 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7017f682-0b04-4bd7-bb5f-612b57ff8543 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:03.006243 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-eb0e91f6-0991-4792-a25c-5519868576f9 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:03.007892 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-eb0e91f6-0991-4792-a25c-5519868576f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:03.039276 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52566d23-8fe5-4b1d-9200-953bdfece0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:03.039913 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52566d23-8fe5-4b1d-9200-953bdfece0c0 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:03.040192 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52566d23-8fe5-4b1d-9200-953bdfece0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:03.040461 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52566d23-8fe5-4b1d-9200-953bdfece0c0 None admin] PATH_INFO: `/v3/projects/fbb3348b420b4f6ba5d171c387e5a403` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:03.041566 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52566d23-8fe5-4b1d-9200-953bdfece0c0 None admin] RBAC: Authorizing `identity:get_project(project_id=fbb3348b420b4f6ba5d171c387e5a403)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:03.043600 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52566d23-8fe5-4b1d-9200-953bdfece0c0 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:03.045552 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 120/249] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:28:02 2022] GET /identity/v3/projects/fbb3348b420b4f6ba5d171c387e5a403 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:03.046706 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02422b12-e822-4673-aca6-985a6cb8038a None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:03.047008 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02422b12-e822-4673-aca6-985a6cb8038a None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:03.047280 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02422b12-e822-4673-aca6-985a6cb8038a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:03.065071 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-02422b12-e822-4673-aca6-985a6cb8038a None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:03.066290 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-02422b12-e822-4673-aca6-985a6cb8038a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:03.076327 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:03.086995 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 130/250] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:03.087745 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e91f6-0991-4792-a25c-5519868576f9 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:03.164530 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 121/251] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:03.165529 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02422b12-e822-4673-aca6-985a6cb8038a None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:03.228743 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e91f6-0991-4792-a25c-5519868576f9 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:03.234242 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:03.268724 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0677cde2-7cae-4ac0-ad59-7fbe6b8974e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:03.269370 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0677cde2-7cae-4ac0-ad59-7fbe6b8974e2 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:03.269662 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0677cde2-7cae-4ac0-ad59-7fbe6b8974e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:03.269981 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0677cde2-7cae-4ac0-ad59-7fbe6b8974e2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:03.284251 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0677cde2-7cae-4ac0-ad59-7fbe6b8974e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:03.289411 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-0677cde2-7cae-4ac0-ad59-7fbe6b8974e2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:28:03.289411 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:03.289411 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:03.289411 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:03.289411 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:03.289411 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:03.289411 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:03.289411 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:03.289411 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:03.289411 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:03.289411 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:03.289411 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:03.289411 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:03.289411 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:03.289411 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:03.289411 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:03.290333 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:03.290333 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:03.290333 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:03.290333 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:03.290333 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:03.290333 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:03.290333 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:03.290333 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:03.290333 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:03.290333 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:03.290333 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:03.290333 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:03.290333 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:03.290333 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:03.290333 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:03.290333 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:03.291106 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:03.291106 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:03.291106 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:03.291106 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:03.291106 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:03.291106 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:03.291106 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:03.291106 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:03.291106 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:03.291106 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:03.291106 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:28:03.291106 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:03.291106 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 131/252] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:28:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:03.297666 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0677cde2-7cae-4ac0-ad59-7fbe6b8974e2 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:03.314963 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02422b12-e822-4673-aca6-985a6cb8038a None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:03.366214 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:03.367740 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe741ad-5c36-4024-bf8e-381f9225313b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:03.368583 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe741ad-5c36-4024-bf8e-381f9225313b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:03.369088 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe741ad-5c36-4024-bf8e-381f9225313b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:03.369496 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe741ad-5c36-4024-bf8e-381f9225313b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:03.371093 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe741ad-5c36-4024-bf8e-381f9225313b None admin] RBAC: Adding query filter params (role.id=9e4cabcdb1e349c699da538ce00d1396, scope.project.id=fbb3348b420b4f6ba5d171c387e5a403, user.id=6c933b0818414e45a45823374f8e61e1) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:03.371544 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe741ad-5c36-4024-bf8e-381f9225313b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:03.373210 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe741ad-5c36-4024-bf8e-381f9225313b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:03.379951 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 122/253] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 15:28:03 2022] GET /identity/v3/role_assignments?user.id=6c933b0818414e45a45823374f8e61e1&scope.project.id=fbb3348b420b4f6ba5d171c387e5a403&role.id=9e4cabcdb1e349c699da538ce00d1396 => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:03.415812 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0677cde2-7cae-4ac0-ad59-7fbe6b8974e2 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:03.454130 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ad3000-6970-44b1-92a5-38eab2d22527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:03.455061 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad3000-6970-44b1-92a5-38eab2d22527 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:03.455343 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad3000-6970-44b1-92a5-38eab2d22527 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:03.455617 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad3000-6970-44b1-92a5-38eab2d22527 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:03.456288 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ad3000-6970-44b1-92a5-38eab2d22527 None admin] RBAC: Adding query filter params (name=member) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:03.456591 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ad3000-6970-44b1-92a5-38eab2d22527 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:03.458036 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ad3000-6970-44b1-92a5-38eab2d22527 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:03.467346 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 132/254] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 15:28:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:28:03.475193 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe741ad-5c36-4024-bf8e-381f9225313b None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:03.617593 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe741ad-5c36-4024-bf8e-381f9225313b None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:03.628411 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:03.637246 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ad3000-6970-44b1-92a5-38eab2d22527 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:03.700090 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d44f912-34a3-4203-a496-eda88c556859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:03.700741 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d44f912-34a3-4203-a496-eda88c556859 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:03.701015 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d44f912-34a3-4203-a496-eda88c556859 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:03.701283 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d44f912-34a3-4203-a496-eda88c556859 None admin] PATH_INFO: `/v3/users/0b8b86dee823406ba22c5057462816c0` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:03.704251 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d44f912-34a3-4203-a496-eda88c556859 None admin] RBAC: Authorizing `identity:get_user(user_id=0b8b86dee823406ba22c5057462816c0)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:03.705718 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d44f912-34a3-4203-a496-eda88c556859 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:03.710979 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 123/255] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:28:03 2022] GET /identity/v3/users/0b8b86dee823406ba22c5057462816c0 => generated 297 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:03.711789 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d44f912-34a3-4203-a496-eda88c556859 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:03.812138 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ad3000-6970-44b1-92a5-38eab2d22527 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:03.840998 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:03.855283 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52334c6-046b-45a5-b783-2fbc04f4fb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:03.855861 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52334c6-046b-45a5-b783-2fbc04f4fb12 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:03.856152 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52334c6-046b-45a5-b783-2fbc04f4fb12 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:03.856424 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52334c6-046b-45a5-b783-2fbc04f4fb12 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:03.866549 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52334c6-046b-45a5-b783-2fbc04f4fb12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:03.872325 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-f52334c6-046b-45a5-b783-2fbc04f4fb12 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:28:03.872325 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:03.872325 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:03.872325 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:03.872325 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:03.872325 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:03.872325 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:03.872325 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:03.872325 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:03.872325 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:03.872325 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:03.872325 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:03.872325 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:03.872325 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:03.872325 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:03.872325 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:03.873201 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:03.873201 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:03.873201 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:03.873201 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:03.873201 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:03.873201 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:03.873201 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:03.873201 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:03.873201 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:03.873201 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:03.873201 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:03.873201 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:03.873201 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:03.873201 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:03.873201 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:03.873201 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:03.874019 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:03.874019 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:03.874019 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:03.874019 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:03.874019 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:03.874019 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:03.874019 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:03.874019 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:03.874019 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:03.874019 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:03.874019 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:28:03.874019 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:03.874019 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 133/256] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 15:28:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:28:03.874661 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52334c6-046b-45a5-b783-2fbc04f4fb12 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:03.907731 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d44f912-34a3-4203-a496-eda88c556859 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:03.973210 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5796ecaf-a1eb-49a6-aedc-f97853bdc503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:03.973841 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5796ecaf-a1eb-49a6-aedc-f97853bdc503 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:03.974353 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5796ecaf-a1eb-49a6-aedc-f97853bdc503 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:03.974621 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5796ecaf-a1eb-49a6-aedc-f97853bdc503 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:03.998068 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5796ecaf-a1eb-49a6-aedc-f97853bdc503 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:04.003445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-5796ecaf-a1eb-49a6-aedc-f97853bdc503 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:28:04.003445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:04.003445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:04.003445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:04.003445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:04.003445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:04.003445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:04.003445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:04.003445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:04.003445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:04.003445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:04.003445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:04.003445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:04.003445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:04.003445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:04.003445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:04.004302 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:04.004302 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:04.004302 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:04.004302 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:04.004302 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:04.004302 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:04.004302 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:04.004302 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:04.004302 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:04.004302 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:04.004302 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:04.004302 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:04.004302 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:04.004302 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:04.004302 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:04.004302 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:04.005082 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:04.005082 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:04.005082 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:04.005082 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:04.005082 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:04.005082 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:04.005082 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:04.005082 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:04.005082 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:04.005082 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:04.005082 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:28:04.005082 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:04.005082 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 124/257] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 15:28:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:04.005714 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5796ecaf-a1eb-49a6-aedc-f97853bdc503 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:04.051720 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52334c6-046b-45a5-b783-2fbc04f4fb12 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:04.071594 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:04.104232 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f0483b-bfea-4018-b4f4-1eaf1b83635a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:04.104911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0483b-bfea-4018-b4f4-1eaf1b83635a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:04.105278 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0483b-bfea-4018-b4f4-1eaf1b83635a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:04.105607 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0483b-bfea-4018-b4f4-1eaf1b83635a None admin] PATH_INFO: `/v3/projects/fbb3348b420b4f6ba5d171c387e5a403` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:04.107089 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f0483b-bfea-4018-b4f4-1eaf1b83635a None admin] RBAC: Authorizing `identity:get_project(project_id=fbb3348b420b4f6ba5d171c387e5a403)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:04.108531 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f0483b-bfea-4018-b4f4-1eaf1b83635a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:04.111907 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 134/258] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:28:03 2022] GET /identity/v3/projects/fbb3348b420b4f6ba5d171c387e5a403 => generated 302 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:04.112520 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f0483b-bfea-4018-b4f4-1eaf1b83635a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:04.205059 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5796ecaf-a1eb-49a6-aedc-f97853bdc503 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:04.213565 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:04.268899 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1887678a-2903-4a42-9438-381248f77a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:04.269806 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887678a-2903-4a42-9438-381248f77a37 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:04.270099 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887678a-2903-4a42-9438-381248f77a37 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:04.272940 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887678a-2903-4a42-9438-381248f77a37 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:04.272940 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1887678a-2903-4a42-9438-381248f77a37 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:04.272940 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1887678a-2903-4a42-9438-381248f77a37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:04.272940 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1887678a-2903-4a42-9438-381248f77a37 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:04.281915 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 125/259] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:28:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:04.282632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1887678a-2903-4a42-9438-381248f77a37 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:04.338260 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f0483b-bfea-4018-b4f4-1eaf1b83635a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:04.410291 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1887678a-2903-4a42-9438-381248f77a37 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:04.413670 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:04.450612 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2d4989-4c50-4327-a06e-d319e4ce6f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:04.452554 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2d4989-4c50-4327-a06e-d319e4ce6f52 None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:04.453043 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2d4989-4c50-4327-a06e-d319e4ce6f52 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:04.454519 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2d4989-4c50-4327-a06e-d319e4ce6f52 None admin] PATH_INFO: `/v3/projects/fbb3348b420b4f6ba5d171c387e5a403/users/0b8b86dee823406ba22c5057462816c0/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:04.458823 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2d4989-4c50-4327-a06e-d319e4ce6f52 None admin] RBAC: Authorizing `identity:create_grant(project_id=fbb3348b420b4f6ba5d171c387e5a403, user_id=0b8b86dee823406ba22c5057462816c0, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:04.460446 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2d4989-4c50-4327-a06e-d319e4ce6f52 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:04.465435 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d18103-01c1-4a7b-8eca-1efd758b285c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:04.466656 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d18103-01c1-4a7b-8eca-1efd758b285c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:04.467122 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d18103-01c1-4a7b-8eca-1efd758b285c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:04.467575 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d18103-01c1-4a7b-8eca-1efd758b285c None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:04.468411 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d18103-01c1-4a7b-8eca-1efd758b285c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:04.470038 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d18103-01c1-4a7b-8eca-1efd758b285c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:04.472229 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d18103-01c1-4a7b-8eca-1efd758b285c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:04.481906 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 135/260] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:28:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:04.483302 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d18103-01c1-4a7b-8eca-1efd758b285c None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:04.495672 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 126/261] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 15:28:04 2022] PUT /identity/v3/projects/fbb3348b420b4f6ba5d171c387e5a403/users/0b8b86dee823406ba22c5057462816c0/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:28:04.497209 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2d4989-4c50-4327-a06e-d319e4ce6f52 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:04.659893 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d18103-01c1-4a7b-8eca-1efd758b285c None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:04.675454 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:04.698802 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ce535f-91c5-4d50-b56f-4fb9269d39ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:04.699763 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce535f-91c5-4d50-b56f-4fb9269d39ba None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:04.700221 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce535f-91c5-4d50-b56f-4fb9269d39ba None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:04.700690 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce535f-91c5-4d50-b56f-4fb9269d39ba None admin] PATH_INFO: `/v3/users/91c065b6fe104421a64d9eb474a7761c` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:04.702156 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ce535f-91c5-4d50-b56f-4fb9269d39ba None admin] RBAC: Authorizing `identity:get_user(user_id=91c065b6fe104421a64d9eb474a7761c)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:04.703930 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ce535f-91c5-4d50-b56f-4fb9269d39ba None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:04.705942 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 136/262] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:28:04 2022] GET /identity/v3/users/91c065b6fe104421a64d9eb474a7761c => generated 283 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:04.714228 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ce535f-91c5-4d50-b56f-4fb9269d39ba None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:04.731850 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2d4989-4c50-4327-a06e-d319e4ce6f52 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:04.803930 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1031ba17-290e-434b-a75a-0c12c4bf618b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:04.804566 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1031ba17-290e-434b-a75a-0c12c4bf618b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:04.804871 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1031ba17-290e-434b-a75a-0c12c4bf618b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:04.805108 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1031ba17-290e-434b-a75a-0c12c4bf618b None admin] PATH_INFO: `/v3/users/91c065b6fe104421a64d9eb474a7761c` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:04.806353 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1031ba17-290e-434b-a75a-0c12c4bf618b None admin] RBAC: Authorizing `identity:get_user(user_id=91c065b6fe104421a64d9eb474a7761c)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:04.808541 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1031ba17-290e-434b-a75a-0c12c4bf618b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:04.810366 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 127/263] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:28:04 2022] GET /identity/v3/users/91c065b6fe104421a64d9eb474a7761c => generated 283 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:04.815604 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1031ba17-290e-434b-a75a-0c12c4bf618b None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:04.845250 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:04.909620 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ce535f-91c5-4d50-b56f-4fb9269d39ba None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:04.922337 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1031ba17-290e-434b-a75a-0c12c4bf618b None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:04.959495 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b86783-38f8-4aff-87ad-c68e3798bf4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:04.960391 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b86783-38f8-4aff-87ad-c68e3798bf4d None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:04.960882 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b86783-38f8-4aff-87ad-c68e3798bf4d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:04.961387 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b86783-38f8-4aff-87ad-c68e3798bf4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:04.967711 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494d08ae-3c69-43ae-8e13-86be8cf03041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:04.968235 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494d08ae-3c69-43ae-8e13-86be8cf03041 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:04.968515 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494d08ae-3c69-43ae-8e13-86be8cf03041 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:04.968789 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494d08ae-3c69-43ae-8e13-86be8cf03041 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:05.033948 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:05.034443 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b86783-38f8-4aff-87ad-c68e3798bf4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:05.035926 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b86783-38f8-4aff-87ad-c68e3798bf4d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:05.063798 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:05.086708 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494d08ae-3c69-43ae-8e13-86be8cf03041 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:05.088199 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494d08ae-3c69-43ae-8e13-86be8cf03041 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:05.218794 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 128/264] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:28:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 28 15:28:05.222907 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b86783-38f8-4aff-87ad-c68e3798bf4d None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:05.278061 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 137/265] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:28:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:28:05.287426 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494d08ae-3c69-43ae-8e13-86be8cf03041 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:05.455350 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b86783-38f8-4aff-87ad-c68e3798bf4d None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:05.455976 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:05.492375 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:05.511994 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2babd68-814c-4f5b-a483-5b9b83c09520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:05.512618 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2babd68-814c-4f5b-a483-5b9b83c09520 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:05.512957 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2babd68-814c-4f5b-a483-5b9b83c09520 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:05.513234 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2babd68-814c-4f5b-a483-5b9b83c09520 None admin] PATH_INFO: `/v3/projects/fbb3348b420b4f6ba5d171c387e5a403` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:05.519187 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2babd68-814c-4f5b-a483-5b9b83c09520 None admin] RBAC: Authorizing `identity:get_project(project_id=fbb3348b420b4f6ba5d171c387e5a403)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:05.520511 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2babd68-814c-4f5b-a483-5b9b83c09520 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:05.522580 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 129/266] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:28:05 2022] GET /identity/v3/projects/fbb3348b420b4f6ba5d171c387e5a403 => generated 302 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:05.531030 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2babd68-814c-4f5b-a483-5b9b83c09520 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:05.532553 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494d08ae-3c69-43ae-8e13-86be8cf03041 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:05.615642 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1451838f-b60b-4d24-be2d-a2b5353942ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:05.616283 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451838f-b60b-4d24-be2d-a2b5353942ad None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:05.616563 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451838f-b60b-4d24-be2d-a2b5353942ad None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:05.616831 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451838f-b60b-4d24-be2d-a2b5353942ad None admin] PATH_INFO: `/v3/projects/fbb3348b420b4f6ba5d171c387e5a403` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:05.617985 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1451838f-b60b-4d24-be2d-a2b5353942ad None admin] RBAC: Authorizing `identity:get_project(project_id=fbb3348b420b4f6ba5d171c387e5a403)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:05.619503 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1451838f-b60b-4d24-be2d-a2b5353942ad None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:05.621326 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 138/267] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:28:05 2022] GET /identity/v3/projects/fbb3348b420b4f6ba5d171c387e5a403 => generated 302 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:05.629576 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1451838f-b60b-4d24-be2d-a2b5353942ad None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:05.699427 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2babd68-814c-4f5b-a483-5b9b83c09520 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:05.742130 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c51da5a-1c19-4d5d-a88a-1bf7074330f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:05.742803 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c51da5a-1c19-4d5d-a88a-1bf7074330f1 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:05.743027 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c51da5a-1c19-4d5d-a88a-1bf7074330f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:05.743301 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c51da5a-1c19-4d5d-a88a-1bf7074330f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:05.744035 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c51da5a-1c19-4d5d-a88a-1bf7074330f1 None admin] RBAC: Adding query filter params (role.id=5038c8140b7145ceb7494210a5c9f9f0, scope.project.id=fbb3348b420b4f6ba5d171c387e5a403, user.id=91c065b6fe104421a64d9eb474a7761c) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:05.744345 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c51da5a-1c19-4d5d-a88a-1bf7074330f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:05.745982 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c51da5a-1c19-4d5d-a88a-1bf7074330f1 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:05.760971 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 130/268] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 15:28:05 2022] GET /identity/v3/role_assignments?user.id=91c065b6fe104421a64d9eb474a7761c&scope.project.id=fbb3348b420b4f6ba5d171c387e5a403&role.id=5038c8140b7145ceb7494210a5c9f9f0 => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:05.762263 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81612602-882d-4568-8b88-aa18cd6760ff None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:05.762548 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81612602-882d-4568-8b88-aa18cd6760ff None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:05.762822 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81612602-882d-4568-8b88-aa18cd6760ff None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:05.764259 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 131/269] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:05.770378 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3eba28-0ec0-4718-826f-ea86919b7632 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:05.770967 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3eba28-0ec0-4718-826f-ea86919b7632 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:05.771427 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3eba28-0ec0-4718-826f-ea86919b7632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:05.805666 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-0c3eba28-0ec0-4718-826f-ea86919b7632 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:05.807397 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-0c3eba28-0ec0-4718-826f-ea86919b7632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:05.886438 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1451838f-b60b-4d24-be2d-a2b5353942ad None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:05.937271 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:05.941049 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fe70bb-1ea1-450c-aea2-34f311d3e1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:05.941972 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fe70bb-1ea1-450c-aea2-34f311d3e1d0 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:05.942509 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fe70bb-1ea1-450c-aea2-34f311d3e1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:05.942969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fe70bb-1ea1-450c-aea2-34f311d3e1d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:05.943919 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fe70bb-1ea1-450c-aea2-34f311d3e1d0 None admin] RBAC: Adding query filter params (role.id=62d96ac9513c452894fc67b442cfc59f, scope.project.id=fbb3348b420b4f6ba5d171c387e5a403, user.id=91c065b6fe104421a64d9eb474a7761c) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:05.944425 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fe70bb-1ea1-450c-aea2-34f311d3e1d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:05.946161 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fe70bb-1ea1-450c-aea2-34f311d3e1d0 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:05.958944 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 139/270] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 15:28:05 2022] GET /identity/v3/role_assignments?user.id=91c065b6fe104421a64d9eb474a7761c&scope.project.id=fbb3348b420b4f6ba5d171c387e5a403&role.id=62d96ac9513c452894fc67b442cfc59f => generated 608 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:05.960818 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ef56c3-554e-40e1-83d7-2572611ed263 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:05.962485 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ef56c3-554e-40e1-83d7-2572611ed263 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:05.965311 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ef56c3-554e-40e1-83d7-2572611ed263 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:05.967629 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 140/271] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:05.970299 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 132/272] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:05.980338 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fdaaf7-a4fa-4900-bf30-2f27acdf1d95 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:05.980998 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fdaaf7-a4fa-4900-bf30-2f27acdf1d95 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:05.981470 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fdaaf7-a4fa-4900-bf30-2f27acdf1d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:05.996018 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:06.003710 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-54fdaaf7-a4fa-4900-bf30-2f27acdf1d95 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:06.005329 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-54fdaaf7-a4fa-4900-bf30-2f27acdf1d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:06.083954 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 141/273] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:06.379871 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3eba28-0ec0-4718-826f-ea86919b7632 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:06.413946 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce605044-2bdb-4703-986c-baa1b17d0aca None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:06.414580 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce605044-2bdb-4703-986c-baa1b17d0aca None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:06.415041 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce605044-2bdb-4703-986c-baa1b17d0aca None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:06.417463 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 142/274] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:06.422708 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd603a6b-4832-4193-9d65-51421daf3779 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:06.423218 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd603a6b-4832-4193-9d65-51421daf3779 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:06.423770 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd603a6b-4832-4193-9d65-51421daf3779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:06.446911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-dd603a6b-4832-4193-9d65-51421daf3779 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:06.449358 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-dd603a6b-4832-4193-9d65-51421daf3779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:06.496586 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3eba28-0ec0-4718-826f-ea86919b7632 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:06.529247 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 143/275] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:28:06.530130 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd603a6b-4832-4193-9d65-51421daf3779 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:06.536833 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23faa319-6557-43a9-bfe9-ac80bde97131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:06.538118 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23faa319-6557-43a9-bfe9-ac80bde97131 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:06.538731 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23faa319-6557-43a9-bfe9-ac80bde97131 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:06.539071 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23faa319-6557-43a9-bfe9-ac80bde97131 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:06.540256 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23faa319-6557-43a9-bfe9-ac80bde97131 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:06.542418 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23faa319-6557-43a9-bfe9-ac80bde97131 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:06.544191 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 133/276] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:06.549678 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23faa319-6557-43a9-bfe9-ac80bde97131 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:06.601691 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:06.628183 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:06.649265 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd603a6b-4832-4193-9d65-51421daf3779 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:06.677053 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23faa319-6557-43a9-bfe9-ac80bde97131 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:06.693786 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c761e0-e6c9-4547-9cbd-e159c0566a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:06.694916 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c761e0-e6c9-4547-9cbd-e159c0566a62 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:06.695378 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c761e0-e6c9-4547-9cbd-e159c0566a62 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:06.696229 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c761e0-e6c9-4547-9cbd-e159c0566a62 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:06.710696 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c761e0-e6c9-4547-9cbd-e159c0566a62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:06.714759 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f297ab22-f0a8-4537-920a-cfab75869ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:06.715485 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297ab22-f0a8-4537-920a-cfab75869ceb None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:06.715831 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297ab22-f0a8-4537-920a-cfab75869ceb None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:06.716170 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297ab22-f0a8-4537-920a-cfab75869ceb None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:06.716889 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f297ab22-f0a8-4537-920a-cfab75869ceb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:06.718076 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-92c761e0-e6c9-4547-9cbd-e159c0566a62 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:28:06.718076 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:06.718076 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:06.718076 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:06.718076 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:06.718076 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:06.718076 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:06.718076 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:06.718076 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:06.718076 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:06.718076 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:06.718076 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:06.718076 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:06.718076 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:06.718076 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:06.718076 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:06.719012 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:06.719012 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:06.719012 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:06.719012 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:06.719012 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:06.719012 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:06.719012 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:06.719012 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:06.719012 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:06.719012 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:06.719012 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:06.719012 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:06.719012 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:06.719012 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:06.719012 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:06.719012 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:06.719832 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:06.719832 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:06.719832 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:06.719832 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:06.719832 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:06.719832 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:06.719832 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:06.719832 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:06.719832 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:06.719832 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:06.719832 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:28:06.719832 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:06.720444 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f297ab22-f0a8-4537-920a-cfab75869ceb None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:06.721937 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 144/277] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:28:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:06.727951 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c761e0-e6c9-4547-9cbd-e159c0566a62 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:06.754296 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 134/278] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 15:28:06 2022] POST /identity/v3/users => generated 293 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:28:06.824753 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f297ab22-f0a8-4537-920a-cfab75869ceb None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:06.850498 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c761e0-e6c9-4547-9cbd-e159c0566a62 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:06.890322 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:06.903749 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b10a5-7710-4f97-9353-e111c5507c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:06.904965 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b10a5-7710-4f97-9353-e111c5507c08 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:06.905394 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b10a5-7710-4f97-9353-e111c5507c08 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:06.906170 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b10a5-7710-4f97-9353-e111c5507c08 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:06.906884 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4b10a5-7710-4f97-9353-e111c5507c08 None admin] RBAC: Adding query filter params (name=member) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:06.907873 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4b10a5-7710-4f97-9353-e111c5507c08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:06.908605 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4b10a5-7710-4f97-9353-e111c5507c08 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:06.923271 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 145/279] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 15:28:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:28:06.924618 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b10a5-7710-4f97-9353-e111c5507c08 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:06.951617 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f297ab22-f0a8-4537-920a-cfab75869ceb None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:06.973000 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:07.009308 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28955e2-cfc4-4ddc-b999-74e66b29cf5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:07.010966 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28955e2-cfc4-4ddc-b999-74e66b29cf5f None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:07.011323 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28955e2-cfc4-4ddc-b999-74e66b29cf5f None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:07.011673 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28955e2-cfc4-4ddc-b999-74e66b29cf5f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:07.028060 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28955e2-cfc4-4ddc-b999-74e66b29cf5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:07.033603 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-f28955e2-cfc4-4ddc-b999-74e66b29cf5f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 15:28:07.033603 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:07.033603 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:07.033603 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:07.033603 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:07.033603 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:07.033603 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:07.033603 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:07.033603 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:07.033603 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:07.033603 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:07.033603 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:07.033603 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:07.033603 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:07.033603 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:07.033603 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:07.034505 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:07.034505 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:07.034505 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:07.034505 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:07.034505 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:07.034505 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:07.034505 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:07.034505 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:07.034505 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:07.034505 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:07.034505 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:07.034505 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:07.034505 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:07.034505 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:07.034505 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:07.034505 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:07.035284 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:07.035284 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:07.035284 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:07.035284 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:07.035284 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:07.035284 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:07.035284 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:07.035284 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:07.035284 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:07.035284 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:07.035284 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 15:28:07.035284 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:07.035871 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 135/280] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:28:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:07.041002 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28955e2-cfc4-4ddc-b999-74e66b29cf5f None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:07.070995 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b10a5-7710-4f97-9353-e111c5507c08 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:07.112099 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b939c27-f232-4c3d-b112-343317896de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:07.112948 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b939c27-f232-4c3d-b112-343317896de8 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:07.113389 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b939c27-f232-4c3d-b112-343317896de8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:07.113801 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b939c27-f232-4c3d-b112-343317896de8 None admin] PATH_INFO: `/v3/users/62f378fe850a4cdeacadcfe7a10d324b` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:07.136303 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b939c27-f232-4c3d-b112-343317896de8 None admin] RBAC: Authorizing `identity:get_user(user_id=62f378fe850a4cdeacadcfe7a10d324b)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:07.138033 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b939c27-f232-4c3d-b112-343317896de8 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:07.140226 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 146/281] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:28:06 2022] GET /identity/v3/users/62f378fe850a4cdeacadcfe7a10d324b => generated 293 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:28:07.146152 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b939c27-f232-4c3d-b112-343317896de8 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:07.156679 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28955e2-cfc4-4ddc-b999-74e66b29cf5f None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:07.196582 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb3edd9-e2c2-4e72-8f37-16c62323f645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:07.197203 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3edd9-e2c2-4e72-8f37-16c62323f645 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:07.197571 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3edd9-e2c2-4e72-8f37-16c62323f645 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:07.198383 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3edd9-e2c2-4e72-8f37-16c62323f645 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:07.199049 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb3edd9-e2c2-4e72-8f37-16c62323f645 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:07.199578 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:07.199688 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb3edd9-e2c2-4e72-8f37-16c62323f645 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:07.201869 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb3edd9-e2c2-4e72-8f37-16c62323f645 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:07.211714 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 136/282] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 15:28:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:07.218162 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb3edd9-e2c2-4e72-8f37-16c62323f645 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:07.255184 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:07.267380 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b939c27-f232-4c3d-b112-343317896de8 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:07.320307 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eacb2ff-71a2-48ce-b7dd-8a9a8b53bfd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:07.321148 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eacb2ff-71a2-48ce-b7dd-8a9a8b53bfd7 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:07.321564 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eacb2ff-71a2-48ce-b7dd-8a9a8b53bfd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:07.322026 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eacb2ff-71a2-48ce-b7dd-8a9a8b53bfd7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:07.323425 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eacb2ff-71a2-48ce-b7dd-8a9a8b53bfd7 None admin] RBAC: Adding query filter params (role.id=e152a36b5a1b470b912b1d705462f512, user.id=62f378fe850a4cdeacadcfe7a10d324b, scope.system=all) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:07.323872 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eacb2ff-71a2-48ce-b7dd-8a9a8b53bfd7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:07.325444 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eacb2ff-71a2-48ce-b7dd-8a9a8b53bfd7 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:07.332033 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 147/283] 10.222.0.52 () {56 vars in 1308 bytes} [Wed Sep 28 15:28:07 2022] GET /identity/v3/role_assignments?user.id=62f378fe850a4cdeacadcfe7a10d324b&scope.system=all&role.id=e152a36b5a1b470b912b1d705462f512 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:28:07.356329 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb3edd9-e2c2-4e72-8f37-16c62323f645 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:07.412116 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9b5ed4-c532-4b63-965f-1f36d68a22eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:07.412680 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b5ed4-c532-4b63-965f-1f36d68a22eb None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:07.412953 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b5ed4-c532-4b63-965f-1f36d68a22eb None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:07.413221 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b5ed4-c532-4b63-965f-1f36d68a22eb None admin] PATH_INFO: `/v3/users/6c933b0818414e45a45823374f8e61e1` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:07.418672 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9b5ed4-c532-4b63-965f-1f36d68a22eb None admin] RBAC: Authorizing `identity:get_user(user_id=6c933b0818414e45a45823374f8e61e1)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:07.420159 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9b5ed4-c532-4b63-965f-1f36d68a22eb None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:07.422274 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 137/284] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:28:07 2022] GET /identity/v3/users/6c933b0818414e45a45823374f8e61e1 => generated 297 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:07.428240 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eacb2ff-71a2-48ce-b7dd-8a9a8b53bfd7 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:07.555474 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f17c25-7556-4a17-9c6c-ae2adfd5f38e None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:07.556008 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f17c25-7556-4a17-9c6c-ae2adfd5f38e None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:07.556008 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f17c25-7556-4a17-9c6c-ae2adfd5f38e None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:07.558900 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 138/285] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:07.564139 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252ca00-3505-40b3-a624-b94deecc327c None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:07.564565 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252ca00-3505-40b3-a624-b94deecc327c None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:07.564842 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252ca00-3505-40b3-a624-b94deecc327c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:07.564942 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eacb2ff-71a2-48ce-b7dd-8a9a8b53bfd7 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:07.581114 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:07.602235 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0dbea3-16d2-47ab-9a57-f32e2a810f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:07.602879 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0dbea3-16d2-47ab-9a57-f32e2a810f9b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:07.603637 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0dbea3-16d2-47ab-9a57-f32e2a810f9b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:07.603637 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0dbea3-16d2-47ab-9a57-f32e2a810f9b None admin] PATH_INFO: `/v3/projects/fbb3348b420b4f6ba5d171c387e5a403` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:07.605129 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0dbea3-16d2-47ab-9a57-f32e2a810f9b None admin] RBAC: Authorizing `identity:get_project(project_id=fbb3348b420b4f6ba5d171c387e5a403)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:07.606675 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0dbea3-16d2-47ab-9a57-f32e2a810f9b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:07.610979 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 148/286] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:28:07 2022] GET /identity/v3/projects/fbb3348b420b4f6ba5d171c387e5a403 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:07.619478 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0dbea3-16d2-47ab-9a57-f32e2a810f9b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:07.622923 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-6252ca00-3505-40b3-a624-b94deecc327c None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:07.624134 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-6252ca00-3505-40b3-a624-b94deecc327c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:07.758299 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 139/287] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:28:07.783347 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0dbea3-16d2-47ab-9a57-f32e2a810f9b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:07.821575 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c87ca39-9aed-4f54-b372-c461195b859c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:07.822279 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c87ca39-9aed-4f54-b372-c461195b859c None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:07.822561 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c87ca39-9aed-4f54-b372-c461195b859c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:07.822891 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c87ca39-9aed-4f54-b372-c461195b859c None admin] PATH_INFO: `/v3/projects/fbb3348b420b4f6ba5d171c387e5a403/users/6c933b0818414e45a45823374f8e61e1/roles/9e4cabcdb1e349c699da538ce00d1396` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:07.826975 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c87ca39-9aed-4f54-b372-c461195b859c None admin] RBAC: Authorizing `identity:create_grant(project_id=fbb3348b420b4f6ba5d171c387e5a403, user_id=6c933b0818414e45a45823374f8e61e1, role_id=9e4cabcdb1e349c699da538ce00d1396)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:07.828446 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c87ca39-9aed-4f54-b372-c461195b859c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:07.850479 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 149/288] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 15:28:07 2022] PUT /identity/v3/projects/fbb3348b420b4f6ba5d171c387e5a403/users/6c933b0818414e45a45823374f8e61e1/roles/9e4cabcdb1e349c699da538ce00d1396 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:28:08.086350 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6252ca00-3505-40b3-a624-b94deecc327c None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:08.125104 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:08.202920 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6252ca00-3505-40b3-a624-b94deecc327c None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:08.251197 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd8acb5-f759-4952-b337-2e94b616c695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:08.252287 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8acb5-f759-4952-b337-2e94b616c695 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:08.252885 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8acb5-f759-4952-b337-2e94b616c695 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:08.253360 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8acb5-f759-4952-b337-2e94b616c695 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:08.272235 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd8acb5-f759-4952-b337-2e94b616c695 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:08.284986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-5fd8acb5-f759-4952-b337-2e94b616c695 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:28:08.284986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:08.284986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:08.284986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:08.284986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:08.284986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:08.284986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:08.284986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:08.284986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:08.284986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:08.284986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:08.284986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:08.284986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:08.284986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:08.284986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:08.284986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:08.286445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:08.286445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:08.286445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:08.286445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:08.286445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:08.286445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:08.286445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:08.286445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:08.286445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:08.286445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:08.286445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:08.286445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:08.286445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:08.286445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:08.286445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:08.286445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:08.287858 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:08.287858 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:08.287858 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:08.287858 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:08.287858 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:08.287858 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:08.287858 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:08.287858 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:08.287858 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:08.287858 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:08.287858 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:28:08.287858 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:08.291377 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 140/289] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:28:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:08.303072 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c87ca39-9aed-4f54-b372-c461195b859c None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:08.378794 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:08.436607 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c87ca39-9aed-4f54-b372-c461195b859c None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:08.471148 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c82026-b278-4c3e-b363-967da0d37fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:08.471800 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c82026-b278-4c3e-b363-967da0d37fec None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:08.472142 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c82026-b278-4c3e-b363-967da0d37fec None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:08.472457 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c82026-b278-4c3e-b363-967da0d37fec None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:08.473111 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c82026-b278-4c3e-b363-967da0d37fec None admin] RBAC: Adding query filter params (name=member) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:08.473410 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c82026-b278-4c3e-b363-967da0d37fec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:08.474789 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c82026-b278-4c3e-b363-967da0d37fec None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:08.483954 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 150/290] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 15:28:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:08.489198 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd8acb5-f759-4952-b337-2e94b616c695 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:08.622346 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:08.641395 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd8acb5-f759-4952-b337-2e94b616c695 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:08.707914 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73578341-01e0-4750-a8b4-5164811e394e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:08.708919 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73578341-01e0-4750-a8b4-5164811e394e None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:08.709395 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73578341-01e0-4750-a8b4-5164811e394e None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:08.709902 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73578341-01e0-4750-a8b4-5164811e394e None admin] PATH_INFO: `/v3/users/0b8b86dee823406ba22c5057462816c0` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:08.714676 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73578341-01e0-4750-a8b4-5164811e394e None admin] RBAC: Authorizing `identity:get_user(user_id=0b8b86dee823406ba22c5057462816c0)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:08.716328 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73578341-01e0-4750-a8b4-5164811e394e None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:08.718438 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 141/291] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:28:08 2022] GET /identity/v3/users/0b8b86dee823406ba22c5057462816c0 => generated 297 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:08.723613 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c82026-b278-4c3e-b363-967da0d37fec None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:08.865960 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:08.870555 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c82026-b278-4c3e-b363-967da0d37fec None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:08.916338 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d7daf7-529e-45ea-ab4c-4f8223c3a881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:08.917399 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d7daf7-529e-45ea-ab4c-4f8223c3a881 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:08.917677 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d7daf7-529e-45ea-ab4c-4f8223c3a881 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:08.917972 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d7daf7-529e-45ea-ab4c-4f8223c3a881 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:09.015423 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d7daf7-529e-45ea-ab4c-4f8223c3a881 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:09.016979 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d7daf7-529e-45ea-ab4c-4f8223c3a881 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:09.141201 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 151/292] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:28:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:28:09.141507 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:09.146568 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73578341-01e0-4750-a8b4-5164811e394e None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:09.292227 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73578341-01e0-4750-a8b4-5164811e394e None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:09.342563 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2f914e-f587-408e-9912-ab0741845f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:09.343198 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2f914e-f587-408e-9912-ab0741845f37 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:09.343478 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2f914e-f587-408e-9912-ab0741845f37 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:09.343749 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2f914e-f587-408e-9912-ab0741845f37 None admin] PATH_INFO: `/v3/projects/fbb3348b420b4f6ba5d171c387e5a403` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:09.344905 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2f914e-f587-408e-9912-ab0741845f37 None admin] RBAC: Authorizing `identity:get_project(project_id=fbb3348b420b4f6ba5d171c387e5a403)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:09.346355 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2f914e-f587-408e-9912-ab0741845f37 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:09.347977 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 142/293] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:28:09 2022] GET /identity/v3/projects/fbb3348b420b4f6ba5d171c387e5a403 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:09.353866 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d7daf7-529e-45ea-ab4c-4f8223c3a881 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:09.486951 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d7daf7-529e-45ea-ab4c-4f8223c3a881 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:09.527008 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef123c42-b199-46f6-88d8-04a2b73370da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:09.528064 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef123c42-b199-46f6-88d8-04a2b73370da None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:09.528586 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef123c42-b199-46f6-88d8-04a2b73370da None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:09.529053 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef123c42-b199-46f6-88d8-04a2b73370da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:09.530036 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef123c42-b199-46f6-88d8-04a2b73370da None admin] RBAC: Adding query filter params (role.id=e152a36b5a1b470b912b1d705462f512, scope.project.id=fbb3348b420b4f6ba5d171c387e5a403, user.id=0b8b86dee823406ba22c5057462816c0) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:09.530539 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef123c42-b199-46f6-88d8-04a2b73370da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:09.532200 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef123c42-b199-46f6-88d8-04a2b73370da None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:09.539596 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 152/294] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 15:28:09 2022] GET /identity/v3/role_assignments?user.id=0b8b86dee823406ba22c5057462816c0&scope.project.id=fbb3348b420b4f6ba5d171c387e5a403&role.id=e152a36b5a1b470b912b1d705462f512 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:09.637906 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae97dc-a91a-4d22-b006-a9e76439f201 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:09.638464 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae97dc-a91a-4d22-b006-a9e76439f201 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:09.638464 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae97dc-a91a-4d22-b006-a9e76439f201 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:09.640100 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 143/295] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:09.645336 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a96ffe-e5a6-4821-9947-0bc8b8d4e7b8 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:09.645988 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a96ffe-e5a6-4821-9947-0bc8b8d4e7b8 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:09.646558 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a96ffe-e5a6-4821-9947-0bc8b8d4e7b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:09.674965 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-80a96ffe-e5a6-4821-9947-0bc8b8d4e7b8 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:09.676596 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-80a96ffe-e5a6-4821-9947-0bc8b8d4e7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:09.762115 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:09.775823 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 153/296] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:09.995213 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a29b0e5-4749-43ed-b7c5-93780d97e277 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:09.995465 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a29b0e5-4749-43ed-b7c5-93780d97e277 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:09.995685 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a29b0e5-4749-43ed-b7c5-93780d97e277 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:09.997258 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 144/297] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:10.005372 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1555b8a-c596-4b96-91fb-9d4ad1cec6f2 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:10.005372 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1555b8a-c596-4b96-91fb-9d4ad1cec6f2 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:10.005372 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1555b8a-c596-4b96-91fb-9d4ad1cec6f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:10.021558 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d1555b8a-c596-4b96-91fb-9d4ad1cec6f2 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:10.023039 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d1555b8a-c596-4b96-91fb-9d4ad1cec6f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:10.081937 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a29b0e5-4749-43ed-b7c5-93780d97e277 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:10.097578 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 154/298] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:10.118772 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:10.195024 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a29b0e5-4749-43ed-b7c5-93780d97e277 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:10.198651 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97392ac7-14c0-47a1-b46d-8d21725c76c2 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:10.198938 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97392ac7-14c0-47a1-b46d-8d21725c76c2 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:10.199215 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97392ac7-14c0-47a1-b46d-8d21725c76c2 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:10.200681 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 155/299] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:28:10.206564 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ecc4b8-9068-4860-b20f-7aa6d05e3368 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:10.206841 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ecc4b8-9068-4860-b20f-7aa6d05e3368 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:10.207111 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ecc4b8-9068-4860-b20f-7aa6d05e3368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:10.229411 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-32ecc4b8-9068-4860-b20f-7aa6d05e3368 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:10.231093 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-32ecc4b8-9068-4860-b20f-7aa6d05e3368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:10.247192 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c08c2-caae-48b4-af33-9ffe72670663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:10.247746 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c08c2-caae-48b4-af33-9ffe72670663 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:10.248027 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c08c2-caae-48b4-af33-9ffe72670663 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:10.248295 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c08c2-caae-48b4-af33-9ffe72670663 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:10.259878 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778c08c2-caae-48b4-af33-9ffe72670663 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:10.265292 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-778c08c2-caae-48b4-af33-9ffe72670663 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 15:28:10.265292 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:10.265292 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:10.265292 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:10.265292 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:10.265292 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:10.265292 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:10.265292 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:10.265292 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:10.265292 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:10.265292 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:10.265292 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:10.265292 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:10.265292 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:10.265292 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:10.265292 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:10.266154 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:10.266154 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:10.266154 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:10.266154 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:10.266154 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:10.266154 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:10.266154 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:10.266154 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:10.266154 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:10.266154 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:10.266154 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:10.266154 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:10.266154 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:10.266154 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:10.266154 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:10.266154 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:10.266931 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:10.266931 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:10.266931 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:10.266931 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:10.266931 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:10.266931 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:10.266931 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:10.266931 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:10.266931 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:10.266931 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:10.266931 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 15:28:10.266931 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:10.266931 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 145/300] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:28:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:28:10.277578 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c08c2-caae-48b4-af33-9ffe72670663 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:10.310744 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 156/301] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:28:10.368141 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ecc4b8-9068-4860-b20f-7aa6d05e3368 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:10.387212 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:10.400937 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c08c2-caae-48b4-af33-9ffe72670663 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:10.425204 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:10.439768 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c9689-51ff-4f05-9d8b-295a6d1d698e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:10.440411 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c9689-51ff-4f05-9d8b-295a6d1d698e None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:10.440648 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c9689-51ff-4f05-9d8b-295a6d1d698e None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:10.440898 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c9689-51ff-4f05-9d8b-295a6d1d698e None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:10.441640 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358c9689-51ff-4f05-9d8b-295a6d1d698e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:10.441969 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358c9689-51ff-4f05-9d8b-295a6d1d698e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:10.443469 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358c9689-51ff-4f05-9d8b-295a6d1d698e None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:10.456928 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 146/302] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 15:28:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:10.458049 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c9689-51ff-4f05-9d8b-295a6d1d698e None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:10.479519 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ecc4b8-9068-4860-b20f-7aa6d05e3368 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:10.514442 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19f2709-9326-41ab-b2d8-350cf508e90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:10.515405 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f2709-9326-41ab-b2d8-350cf508e90e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:10.515877 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f2709-9326-41ab-b2d8-350cf508e90e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:10.516346 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f2709-9326-41ab-b2d8-350cf508e90e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:10.527279 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19f2709-9326-41ab-b2d8-350cf508e90e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:10.532442 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-c19f2709-9326-41ab-b2d8-350cf508e90e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:28:10.532442 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:10.532442 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:10.532442 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:10.532442 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:10.532442 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:10.532442 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:10.532442 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:10.532442 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:10.532442 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:10.532442 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:10.532442 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:10.532442 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:10.532442 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:10.532442 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:10.532442 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:10.533292 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:10.533292 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:10.533292 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:10.533292 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:10.533292 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:10.533292 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:10.533292 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:10.533292 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:10.533292 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:10.533292 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:10.533292 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:10.533292 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:10.533292 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:10.533292 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:10.533292 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:10.533292 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:10.534111 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:10.534111 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:10.534111 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:10.534111 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:10.534111 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:10.534111 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:10.534111 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:10.534111 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:10.534111 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:10.534111 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:10.534111 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:28:10.534111 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:10.536321 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 157/303] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:28:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:28:10.544006 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19f2709-9326-41ab-b2d8-350cf508e90e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:10.573295 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c9689-51ff-4f05-9d8b-295a6d1d698e None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:10.617899 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4606f3-3d0d-4c67-bc26-6edaf21bbc9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:10.618634 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4606f3-3d0d-4c67-bc26-6edaf21bbc9b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:10.618912 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4606f3-3d0d-4c67-bc26-6edaf21bbc9b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:10.619213 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4606f3-3d0d-4c67-bc26-6edaf21bbc9b None admin] PATH_INFO: `/v3/users/8137d773fc2c46bd896204082fc8e81c` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:10.633400 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4606f3-3d0d-4c67-bc26-6edaf21bbc9b None admin] RBAC: Authorizing `identity:get_user(user_id=8137d773fc2c46bd896204082fc8e81c)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:10.635138 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4606f3-3d0d-4c67-bc26-6edaf21bbc9b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:10.636794 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 147/304] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:28:10 2022] GET /identity/v3/users/8137d773fc2c46bd896204082fc8e81c => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:28:10.638848 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4606f3-3d0d-4c67-bc26-6edaf21bbc9b None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:10.651399 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:10.654085 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19f2709-9326-41ab-b2d8-350cf508e90e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:10.669222 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:10.690715 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7f34e3-0932-40b8-8171-9de6263cc03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:10.691221 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f34e3-0932-40b8-8171-9de6263cc03e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:10.691497 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f34e3-0932-40b8-8171-9de6263cc03e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:10.691762 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f34e3-0932-40b8-8171-9de6263cc03e None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:10.692415 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7f34e3-0932-40b8-8171-9de6263cc03e None admin] RBAC: Adding query filter params (name=member) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:10.692710 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7f34e3-0932-40b8-8171-9de6263cc03e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:10.694108 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7f34e3-0932-40b8-8171-9de6263cc03e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:10.702741 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 158/305] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 15:28:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:28:10.707091 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7f34e3-0932-40b8-8171-9de6263cc03e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:10.743643 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4606f3-3d0d-4c67-bc26-6edaf21bbc9b None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:10.777010 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc4fb2-657a-4542-8ba5-20609fb62170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:10.777564 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc4fb2-657a-4542-8ba5-20609fb62170 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:10.777837 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc4fb2-657a-4542-8ba5-20609fb62170 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:10.778131 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc4fb2-657a-4542-8ba5-20609fb62170 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:10.779211 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcc4fb2-657a-4542-8ba5-20609fb62170 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:10.780626 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcc4fb2-657a-4542-8ba5-20609fb62170 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:10.782198 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 148/306] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:10.783412 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569efce0-4932-41bf-bdfd-2e7afcf97fa8 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:10.783692 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569efce0-4932-41bf-bdfd-2e7afcf97fa8 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:10.783960 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569efce0-4932-41bf-bdfd-2e7afcf97fa8 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:10.785259 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 149/307] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:10.785885 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569efce0-4932-41bf-bdfd-2e7afcf97fa8 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:10.810968 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7f34e3-0932-40b8-8171-9de6263cc03e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:10.843176 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381b2faf-209b-403d-b3fe-41496a6f1084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:10.843775 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b2faf-209b-403d-b3fe-41496a6f1084 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:10.844052 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b2faf-209b-403d-b3fe-41496a6f1084 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:10.844323 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b2faf-209b-403d-b3fe-41496a6f1084 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:10.845051 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381b2faf-209b-403d-b3fe-41496a6f1084 None admin] RBAC: Adding query filter params (role.id=9e4cabcdb1e349c699da538ce00d1396, user.id=8137d773fc2c46bd896204082fc8e81c, scope.system=all) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:10.845358 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381b2faf-209b-403d-b3fe-41496a6f1084 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:10.846806 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381b2faf-209b-403d-b3fe-41496a6f1084 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:10.851862 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 159/308] 10.222.0.52 () {56 vars in 1308 bytes} [Wed Sep 28 15:28:10 2022] GET /identity/v3/role_assignments?user.id=8137d773fc2c46bd896204082fc8e81c&scope.system=all&role.id=9e4cabcdb1e349c699da538ce00d1396 => generated 226 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:10.852526 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381b2faf-209b-403d-b3fe-41496a6f1084 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:10.870834 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:10.900917 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569efce0-4932-41bf-bdfd-2e7afcf97fa8 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:10.903233 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:10.944158 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d34de94-8c6c-4788-997d-00f98c5a2ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:10.945599 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d34de94-8c6c-4788-997d-00f98c5a2ca4 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:10.946157 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d34de94-8c6c-4788-997d-00f98c5a2ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:10.946576 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d34de94-8c6c-4788-997d-00f98c5a2ca4 None admin] PATH_INFO: `/v3/users/62f378fe850a4cdeacadcfe7a10d324b` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:10.948560 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d34de94-8c6c-4788-997d-00f98c5a2ca4 None admin] RBAC: Authorizing `identity:get_user(user_id=62f378fe850a4cdeacadcfe7a10d324b)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:10.950486 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d34de94-8c6c-4788-997d-00f98c5a2ca4 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:10.953127 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 150/309] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:28:10 2022] GET /identity/v3/users/62f378fe850a4cdeacadcfe7a10d324b => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:10.955035 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f396bcd9-9652-4f83-aa18-fb7774acd97d None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:10.955462 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f396bcd9-9652-4f83-aa18-fb7774acd97d None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:10.955856 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f396bcd9-9652-4f83-aa18-fb7774acd97d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:10.967673 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381b2faf-209b-403d-b3fe-41496a6f1084 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:10.981140 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-f396bcd9-9652-4f83-aa18-fb7774acd97d None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:10.982753 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-f396bcd9-9652-4f83-aa18-fb7774acd97d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:11.006000 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037a46c9-9a65-4799-9f42-8bd74b75881f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:11.006881 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a46c9-9a65-4799-9f42-8bd74b75881f None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:11.007158 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a46c9-9a65-4799-9f42-8bd74b75881f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:11.007428 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a46c9-9a65-4799-9f42-8bd74b75881f None admin] PATH_INFO: `/v3/groups` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:11.008054 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037a46c9-9a65-4799-9f42-8bd74b75881f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:11.009523 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037a46c9-9a65-4799-9f42-8bd74b75881f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:11.028035 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 160/310] 10.222.0.52 () {60 vars in 1134 bytes} [Wed Sep 28 15:28:10 2022] POST /identity/v3/groups => generated 229 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:28:11.028843 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037a46c9-9a65-4799-9f42-8bd74b75881f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:11.084708 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 151/311] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:11.149017 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:11.152401 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037a46c9-9a65-4799-9f42-8bd74b75881f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:11.192812 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af818d6d-c5aa-4880-8b30-c3f0f206e8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:11.193481 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af818d6d-c5aa-4880-8b30-c3f0f206e8b2 None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:11.193762 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af818d6d-c5aa-4880-8b30-c3f0f206e8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:11.194082 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af818d6d-c5aa-4880-8b30-c3f0f206e8b2 None admin] PATH_INFO: `/v3/system/users/62f378fe850a4cdeacadcfe7a10d324b/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:11.195652 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af818d6d-c5aa-4880-8b30-c3f0f206e8b2 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=62f378fe850a4cdeacadcfe7a10d324b, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:11.197038 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af818d6d-c5aa-4880-8b30-c3f0f206e8b2 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:11.208490 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 161/312] 10.222.0.52 () {58 vars in 1334 bytes} [Wed Sep 28 15:28:11 2022] PUT /identity/v3/system/users/62f378fe850a4cdeacadcfe7a10d324b/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:28:11.372588 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f396bcd9-9652-4f83-aa18-fb7774acd97d None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:11.452882 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:11.481008 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f396bcd9-9652-4f83-aa18-fb7774acd97d None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:11.515518 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28622a17-3ae3-4f94-b670-e2fe3c8bc863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:11.516496 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28622a17-3ae3-4f94-b670-e2fe3c8bc863 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:11.516988 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28622a17-3ae3-4f94-b670-e2fe3c8bc863 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:11.517462 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28622a17-3ae3-4f94-b670-e2fe3c8bc863 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:11.528430 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28622a17-3ae3-4f94-b670-e2fe3c8bc863 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:11.534212 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-28622a17-3ae3-4f94-b670-e2fe3c8bc863 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 15:28:11.534212 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:11.534212 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:11.534212 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:11.534212 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:11.534212 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:11.534212 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:11.534212 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:11.534212 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:11.534212 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:11.534212 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:11.534212 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:11.534212 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:11.534212 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:11.534212 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:11.534212 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:11.535095 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:11.535095 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:11.535095 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:11.535095 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:11.535095 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:11.535095 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:11.535095 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:11.535095 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:11.535095 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:11.535095 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:11.535095 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:11.535095 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:11.535095 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:11.535095 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:11.535095 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:11.535095 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:11.535884 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:11.535884 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:11.535884 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:11.535884 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:11.535884 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:11.535884 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:11.535884 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:11.535884 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:11.535884 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:11.535884 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:11.535884 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 15:28:11.535884 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:11.544562 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 152/313] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:28:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:11.546292 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af818d6d-c5aa-4880-8b30-c3f0f206e8b2 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:11.654758 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af818d6d-c5aa-4880-8b30-c3f0f206e8b2 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:11.699855 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8535ff2-5da0-4f77-b619-2394cd7f539f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:11.700819 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8535ff2-5da0-4f77-b619-2394cd7f539f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:11.701317 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8535ff2-5da0-4f77-b619-2394cd7f539f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:11.701784 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8535ff2-5da0-4f77-b619-2394cd7f539f None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:11.702675 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8535ff2-5da0-4f77-b619-2394cd7f539f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:11.703191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8535ff2-5da0-4f77-b619-2394cd7f539f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:11.704786 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8535ff2-5da0-4f77-b619-2394cd7f539f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:11.719504 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 162/314] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 15:28:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:11.721139 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28622a17-3ae3-4f94-b670-e2fe3c8bc863 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:11.826821 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28622a17-3ae3-4f94-b670-e2fe3c8bc863 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:11.862045 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80687005-d652-4ab2-ba79-30341a2afbbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:11.863006 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80687005-d652-4ab2-ba79-30341a2afbbb None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:11.863501 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80687005-d652-4ab2-ba79-30341a2afbbb None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:11.863984 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80687005-d652-4ab2-ba79-30341a2afbbb None admin] PATH_INFO: `/v3/users/6c933b0818414e45a45823374f8e61e1` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:11.865299 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80687005-d652-4ab2-ba79-30341a2afbbb None admin] RBAC: Authorizing `identity:get_user(user_id=6c933b0818414e45a45823374f8e61e1)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:11.866937 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80687005-d652-4ab2-ba79-30341a2afbbb None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:11.868701 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 153/315] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:28:11 2022] GET /identity/v3/users/6c933b0818414e45a45823374f8e61e1 => generated 297 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:11.875103 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8535ff2-5da0-4f77-b619-2394cd7f539f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:11.903856 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:11.979332 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8535ff2-5da0-4f77-b619-2394cd7f539f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:12.017248 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf456c01-bf65-4904-ab84-8d04b97b1edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:12.017875 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf456c01-bf65-4904-ab84-8d04b97b1edf None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:12.018181 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf456c01-bf65-4904-ab84-8d04b97b1edf None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:12.018459 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf456c01-bf65-4904-ab84-8d04b97b1edf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:12.087361 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf456c01-bf65-4904-ab84-8d04b97b1edf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:12.089443 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf456c01-bf65-4904-ab84-8d04b97b1edf None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:12.118183 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:12.195458 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 163/316] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:28:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:28:12.201001 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80687005-d652-4ab2-ba79-30341a2afbbb None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:12.214179 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:12.303505 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80687005-d652-4ab2-ba79-30341a2afbbb None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:12.344868 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a098a0-ba6c-41f7-bebe-11909041d4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:12.345430 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a098a0-ba6c-41f7-bebe-11909041d4f0 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:12.345706 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a098a0-ba6c-41f7-bebe-11909041d4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:12.346007 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a098a0-ba6c-41f7-bebe-11909041d4f0 None admin] PATH_INFO: `/v3/projects/fbb3348b420b4f6ba5d171c387e5a403` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:12.347122 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a098a0-ba6c-41f7-bebe-11909041d4f0 None admin] RBAC: Authorizing `identity:get_project(project_id=fbb3348b420b4f6ba5d171c387e5a403)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:12.348557 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a098a0-ba6c-41f7-bebe-11909041d4f0 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:12.350509 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 154/317] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:28:12 2022] GET /identity/v3/projects/fbb3348b420b4f6ba5d171c387e5a403 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:12.356149 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf456c01-bf65-4904-ab84-8d04b97b1edf None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:12.458826 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf456c01-bf65-4904-ab84-8d04b97b1edf None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:12.484498 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:12.494047 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb4fe35-066f-4bdf-bc65-160fc0f14912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:12.495277 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4fe35-066f-4bdf-bc65-160fc0f14912 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:12.495566 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4fe35-066f-4bdf-bc65-160fc0f14912 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:12.495847 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4fe35-066f-4bdf-bc65-160fc0f14912 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:12.496578 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb4fe35-066f-4bdf-bc65-160fc0f14912 None admin] RBAC: Adding query filter params (role.id=9e4cabcdb1e349c699da538ce00d1396, scope.project.id=fbb3348b420b4f6ba5d171c387e5a403, user.id=6c933b0818414e45a45823374f8e61e1) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:12.496893 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb4fe35-066f-4bdf-bc65-160fc0f14912 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:12.498697 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb4fe35-066f-4bdf-bc65-160fc0f14912 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:12.505453 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 164/318] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 15:28:12 2022] GET /identity/v3/role_assignments?user.id=6c933b0818414e45a45823374f8e61e1&scope.project.id=fbb3348b420b4f6ba5d171c387e5a403&role.id=9e4cabcdb1e349c699da538ce00d1396 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:13.134908 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0a2ee-4a08-47a5-b803-e961c2915c6d None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:13.136381 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0a2ee-4a08-47a5-b803-e961c2915c6d None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:13.136852 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0a2ee-4a08-47a5-b803-e961c2915c6d None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:13.138404 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 155/319] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:13.143758 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d30b8-3ea1-41f3-9fa7-4a9631a342f2 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:13.144144 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d30b8-3ea1-41f3-9fa7-4a9631a342f2 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:13.144316 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d30b8-3ea1-41f3-9fa7-4a9631a342f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:13.163123 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-863d30b8-3ea1-41f3-9fa7-4a9631a342f2 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:13.164308 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-863d30b8-3ea1-41f3-9fa7-4a9631a342f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:13.237761 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 165/320] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:13.293831 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2bcc0-5dc2-4999-b6e5-7b096876f7f9 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:13.294166 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2bcc0-5dc2-4999-b6e5-7b096876f7f9 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:13.294428 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2bcc0-5dc2-4999-b6e5-7b096876f7f9 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:13.295885 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 156/321] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:13.301035 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b2417-fe05-4d7d-9d26-47fb9c5803bf None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:13.301325 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b2417-fe05-4d7d-9d26-47fb9c5803bf None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:13.301598 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b2417-fe05-4d7d-9d26-47fb9c5803bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:13.319683 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-378b2417-fe05-4d7d-9d26-47fb9c5803bf None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:13.320799 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-378b2417-fe05-4d7d-9d26-47fb9c5803bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:13.392512 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 166/322] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:13.453624 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b21287-648e-42fd-9cff-88ed9dc81a2c None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:13.453940 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b21287-648e-42fd-9cff-88ed9dc81a2c None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:13.454201 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b21287-648e-42fd-9cff-88ed9dc81a2c None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:13.455654 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 157/323] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:13.460578 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f44fa-8b19-49b0-85c1-88a04b697438 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:13.460868 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f44fa-8b19-49b0-85c1-88a04b697438 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:13.461143 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f44fa-8b19-49b0-85c1-88a04b697438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:13.479586 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d92f44fa-8b19-49b0-85c1-88a04b697438 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:13.480752 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d92f44fa-8b19-49b0-85c1-88a04b697438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:13.505942 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b21287-648e-42fd-9cff-88ed9dc81a2c None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:13.530393 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:13.555949 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 167/324] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:13.582306 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:13.614597 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b21287-648e-42fd-9cff-88ed9dc81a2c None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:13.651096 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba94b41-8228-4530-b5cb-a1fea90f76fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:13.651637 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba94b41-8228-4530-b5cb-a1fea90f76fa None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:13.651908 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba94b41-8228-4530-b5cb-a1fea90f76fa None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:13.652171 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba94b41-8228-4530-b5cb-a1fea90f76fa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:13.661945 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92f44fa-8b19-49b0-85c1-88a04b697438 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:13.662538 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba94b41-8228-4530-b5cb-a1fea90f76fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:13.668540 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-bba94b41-8228-4530-b5cb-a1fea90f76fa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 15:28:13.668540 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:13.668540 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:13.668540 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:13.668540 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:13.668540 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:13.668540 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:13.668540 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:13.668540 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:13.668540 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:13.668540 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:13.668540 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:13.668540 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:13.668540 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:13.668540 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:13.668540 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:13.669383 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:13.669383 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:13.669383 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:13.669383 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:13.669383 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:13.669383 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:13.669383 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:13.669383 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:13.669383 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:13.669383 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:13.669383 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:13.669383 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:13.669383 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:13.669383 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:13.669383 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:13.669383 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:13.674892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:13.674892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:13.674892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:13.674892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:13.674892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:13.674892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:13.674892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:13.674892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:13.674892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:13.674892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:13.674892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 15:28:13.674892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:13.674892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 158/325] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:28:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:28:13.675559 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba94b41-8228-4530-b5cb-a1fea90f76fa None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:13.761599 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92f44fa-8b19-49b0-85c1-88a04b697438 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:13.796020 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6532cc-d64c-496b-9086-8004919e35a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:13.796982 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6532cc-d64c-496b-9086-8004919e35a7 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:13.797434 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6532cc-d64c-496b-9086-8004919e35a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:13.797899 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6532cc-d64c-496b-9086-8004919e35a7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:13.798887 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba94b41-8228-4530-b5cb-a1fea90f76fa None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:13.800733 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6532cc-d64c-496b-9086-8004919e35a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:13.802463 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6532cc-d64c-496b-9086-8004919e35a7 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:13.804504 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 168/326] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:28:13.810815 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6532cc-d64c-496b-9086-8004919e35a7 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:13.831395 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:13.837528 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c46ed2-8330-45a2-b402-add631fdd142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:13.838285 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c46ed2-8330-45a2-b402-add631fdd142 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:13.838565 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c46ed2-8330-45a2-b402-add631fdd142 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:13.838833 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c46ed2-8330-45a2-b402-add631fdd142 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:13.839491 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c46ed2-8330-45a2-b402-add631fdd142 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:13.839791 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c46ed2-8330-45a2-b402-add631fdd142 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:13.841163 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c46ed2-8330-45a2-b402-add631fdd142 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:13.849125 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 159/327] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 15:28:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:28:13.850716 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c46ed2-8330-45a2-b402-add631fdd142 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:13.860367 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:13.910698 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6532cc-d64c-496b-9086-8004919e35a7 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:13.944295 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b7f804-bee7-4386-8975-39954219ad77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:13.944902 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7f804-bee7-4386-8975-39954219ad77 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:13.945178 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7f804-bee7-4386-8975-39954219ad77 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:13.945449 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b7f804-bee7-4386-8975-39954219ad77 None admin] PATH_INFO: `/v3/groups` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:13.946100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b7f804-bee7-4386-8975-39954219ad77 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:13.947290 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c46ed2-8330-45a2-b402-add631fdd142 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:13.947558 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b7f804-bee7-4386-8975-39954219ad77 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:13.969909 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 169/328] 10.222.0.52 () {60 vars in 1134 bytes} [Wed Sep 28 15:28:13 2022] POST /identity/v3/groups => generated 226 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:28:13.969909 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b7f804-bee7-4386-8975-39954219ad77 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:13.985381 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868268b5-41ea-4792-afe8-4ed9e8e6549d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:13.986066 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868268b5-41ea-4792-afe8-4ed9e8e6549d None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:13.986340 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868268b5-41ea-4792-afe8-4ed9e8e6549d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:13.986610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868268b5-41ea-4792-afe8-4ed9e8e6549d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:13.999788 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868268b5-41ea-4792-afe8-4ed9e8e6549d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:14.005206 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-868268b5-41ea-4792-afe8-4ed9e8e6549d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:28:14.005206 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:14.005206 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:14.005206 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:14.005206 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:14.005206 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:14.005206 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:14.005206 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:14.005206 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:14.005206 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:14.005206 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:14.005206 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:14.005206 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:14.005206 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:14.005206 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:14.005206 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:14.006651 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:14.006651 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:14.006651 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:14.006651 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:14.006651 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:14.006651 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:14.006651 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:14.006651 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:14.006651 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:14.006651 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:14.006651 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:14.006651 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:14.006651 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:14.006651 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:14.006651 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:14.006651 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:14.007452 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:14.007452 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:14.007452 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:14.007452 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:14.007452 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:14.007452 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:14.007452 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:14.007452 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:14.007452 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:14.007452 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:14.007452 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:28:14.007452 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:14.007452 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 160/329] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:28:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:14.013045 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868268b5-41ea-4792-afe8-4ed9e8e6549d None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:14.075274 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:14.086589 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b7f804-bee7-4386-8975-39954219ad77 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:14.099073 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:14.144712 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868268b5-41ea-4792-afe8-4ed9e8e6549d None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:14.146791 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61585eb-3da0-477b-8425-061f135772fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:14.147748 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61585eb-3da0-477b-8425-061f135772fd None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:14.148172 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61585eb-3da0-477b-8425-061f135772fd None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:14.148595 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61585eb-3da0-477b-8425-061f135772fd None admin] PATH_INFO: `/v3/users/8137d773fc2c46bd896204082fc8e81c` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:14.150325 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61585eb-3da0-477b-8425-061f135772fd None admin] RBAC: Authorizing `identity:get_user(user_id=8137d773fc2c46bd896204082fc8e81c)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:14.153210 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61585eb-3da0-477b-8425-061f135772fd None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:14.155743 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 170/330] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:28:13 2022] GET /identity/v3/users/8137d773fc2c46bd896204082fc8e81c => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:14.161899 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61585eb-3da0-477b-8425-061f135772fd None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:14.186302 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0665b33-2f0f-48e9-9674-9c725f8d6f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:14.186912 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0665b33-2f0f-48e9-9674-9c725f8d6f7d None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:14.187189 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0665b33-2f0f-48e9-9674-9c725f8d6f7d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:14.187457 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0665b33-2f0f-48e9-9674-9c725f8d6f7d None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:14.188138 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0665b33-2f0f-48e9-9674-9c725f8d6f7d None admin] RBAC: Adding query filter params (name=member) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:14.188420 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0665b33-2f0f-48e9-9674-9c725f8d6f7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:14.189800 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0665b33-2f0f-48e9-9674-9c725f8d6f7d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:14.200122 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 161/331] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 15:28:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:28:14.205809 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0665b33-2f0f-48e9-9674-9c725f8d6f7d None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:14.292356 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61585eb-3da0-477b-8425-061f135772fd None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:14.335093 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848553a6-f6e6-44c0-a339-acaa7c8fdc1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:14.335751 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848553a6-f6e6-44c0-a339-acaa7c8fdc1e None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:14.335848 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0665b33-2f0f-48e9-9674-9c725f8d6f7d None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:14.336027 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848553a6-f6e6-44c0-a339-acaa7c8fdc1e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:14.336350 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848553a6-f6e6-44c0-a339-acaa7c8fdc1e None admin] PATH_INFO: `/v3/system/users/8137d773fc2c46bd896204082fc8e81c/roles/9e4cabcdb1e349c699da538ce00d1396` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:14.338064 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848553a6-f6e6-44c0-a339-acaa7c8fdc1e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8137d773fc2c46bd896204082fc8e81c, role_id=9e4cabcdb1e349c699da538ce00d1396)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:14.339476 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848553a6-f6e6-44c0-a339-acaa7c8fdc1e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:14.347696 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 171/332] 10.222.0.52 () {58 vars in 1334 bytes} [Wed Sep 28 15:28:14 2022] PUT /identity/v3/system/users/8137d773fc2c46bd896204082fc8e81c/roles/9e4cabcdb1e349c699da538ce00d1396 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:28:14.355575 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:14.376790 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fb08df-d075-4e3c-bc1d-65e7b4ed267d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:14.377442 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb08df-d075-4e3c-bc1d-65e7b4ed267d None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:14.377716 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb08df-d075-4e3c-bc1d-65e7b4ed267d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:14.378021 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb08df-d075-4e3c-bc1d-65e7b4ed267d None admin] PATH_INFO: `/v3/users/62f378fe850a4cdeacadcfe7a10d324b` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:14.379271 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fb08df-d075-4e3c-bc1d-65e7b4ed267d None admin] RBAC: Authorizing `identity:get_user(user_id=62f378fe850a4cdeacadcfe7a10d324b)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:14.380777 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fb08df-d075-4e3c-bc1d-65e7b4ed267d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:14.383021 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 162/333] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:28:14 2022] GET /identity/v3/users/62f378fe850a4cdeacadcfe7a10d324b => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:28:14.389557 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848553a6-f6e6-44c0-a339-acaa7c8fdc1e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:14.449008 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:14.524480 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848553a6-f6e6-44c0-a339-acaa7c8fdc1e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:14.563253 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1892193d-ace5-4155-9abd-60f012e7cf80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:14.563862 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892193d-ace5-4155-9abd-60f012e7cf80 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:14.564138 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892193d-ace5-4155-9abd-60f012e7cf80 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:14.564403 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892193d-ace5-4155-9abd-60f012e7cf80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:14.565128 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1892193d-ace5-4155-9abd-60f012e7cf80 None admin] RBAC: Adding query filter params (role.id=e152a36b5a1b470b912b1d705462f512, user.id=62f378fe850a4cdeacadcfe7a10d324b, scope.system=all) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:14.565477 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1892193d-ace5-4155-9abd-60f012e7cf80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:14.566902 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1892193d-ace5-4155-9abd-60f012e7cf80 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:14.580509 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 172/334] 10.222.0.52 () {56 vars in 1308 bytes} [Wed Sep 28 15:28:14 2022] GET /identity/v3/role_assignments?user.id=62f378fe850a4cdeacadcfe7a10d324b&scope.system=all&role.id=e152a36b5a1b470b912b1d705462f512 => generated 510 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:16.365266 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c3fa64-b43e-49d9-8e56-6157e0912fa1 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:16.365794 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c3fa64-b43e-49d9-8e56-6157e0912fa1 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:16.365794 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c3fa64-b43e-49d9-8e56-6157e0912fa1 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:16.367297 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 163/335] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:16.374186 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e3584-9061-49a9-807a-db2bf6aa3eb6 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:16.375065 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e3584-9061-49a9-807a-db2bf6aa3eb6 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:16.375573 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e3584-9061-49a9-807a-db2bf6aa3eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:16.396671 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-155e3584-9061-49a9-807a-db2bf6aa3eb6 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:16.398219 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-155e3584-9061-49a9-807a-db2bf6aa3eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:16.475972 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 173/336] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:28:16.762955 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b0efd-8edc-4985-92f6-36b871439b0d None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:16.763356 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b0efd-8edc-4985-92f6-36b871439b0d None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:16.763528 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b0efd-8edc-4985-92f6-36b871439b0d None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:16.765067 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 164/337] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:16.770217 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d81cd-289d-4b17-b00f-d91e2b741a3c None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:16.770743 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d81cd-289d-4b17-b00f-d91e2b741a3c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:16.771209 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d81cd-289d-4b17-b00f-d91e2b741a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:16.793819 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d30d81cd-289d-4b17-b00f-d91e2b741a3c None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:16.795330 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d30d81cd-289d-4b17-b00f-d91e2b741a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:16.826774 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:16.877449 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 174/338] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:16.891802 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9b0efd-8edc-4985-92f6-36b871439b0d None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:17.038981 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9b0efd-8edc-4985-92f6-36b871439b0d None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:17.080129 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d604df-f95b-463c-981e-b80edfa89f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:17.081111 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d604df-f95b-463c-981e-b80edfa89f7e None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:17.081601 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d604df-f95b-463c-981e-b80edfa89f7e None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:17.082095 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d604df-f95b-463c-981e-b80edfa89f7e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:17.097587 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d604df-f95b-463c-981e-b80edfa89f7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:17.102767 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-98d604df-f95b-463c-981e-b80edfa89f7e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:28:17.102767 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:17.102767 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:17.102767 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:17.102767 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:17.102767 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:17.102767 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:17.102767 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:17.102767 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:17.102767 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:17.102767 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:17.102767 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:17.102767 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:17.102767 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:17.102767 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:17.102767 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:17.103705 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:17.103705 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:17.103705 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:17.103705 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:17.103705 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:17.103705 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:17.103705 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:17.103705 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:17.103705 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:17.103705 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:17.103705 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:17.103705 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:17.103705 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:17.103705 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:17.103705 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:17.103705 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:17.104513 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:17.104513 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:17.104513 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:17.104513 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:17.104513 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:17.104513 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:17.104513 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:17.104513 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:17.104513 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:17.104513 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:17.104513 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:28:17.104513 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:17.108056 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 165/339] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:28:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:17.114132 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30d81cd-289d-4b17-b00f-d91e2b741a3c None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:17.174179 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab727f-55a6-4506-879f-5a91a7729ebe None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:17.174791 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab727f-55a6-4506-879f-5a91a7729ebe None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:17.175258 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab727f-55a6-4506-879f-5a91a7729ebe None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:17.176773 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 166/340] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:28:17.182068 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f943e-13e1-43e9-b6be-da08145df094 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:17.182347 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f943e-13e1-43e9-b6be-da08145df094 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:17.182615 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f943e-13e1-43e9-b6be-da08145df094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:17.211474 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-391f943e-13e1-43e9-b6be-da08145df094 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:17.214113 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-391f943e-13e1-43e9-b6be-da08145df094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:17.276463 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30d81cd-289d-4b17-b00f-d91e2b741a3c None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:17.281720 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:17.342987 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 167/341] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:17.350564 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54ea798-f5dc-43d7-9ed4-a706f8916c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:17.351195 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ea798-f5dc-43d7-9ed4-a706f8916c78 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:17.351478 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ea798-f5dc-43d7-9ed4-a706f8916c78 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:17.351751 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54ea798-f5dc-43d7-9ed4-a706f8916c78 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:17.352417 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54ea798-f5dc-43d7-9ed4-a706f8916c78 None admin] RBAC: Adding query filter params (name=member) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:17.352715 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54ea798-f5dc-43d7-9ed4-a706f8916c78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:17.354167 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54ea798-f5dc-43d7-9ed4-a706f8916c78 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:17.373081 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 175/342] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 15:28:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:17.378752 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391f943e-13e1-43e9-b6be-da08145df094 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:17.455153 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54ea798-f5dc-43d7-9ed4-a706f8916c78 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:17.493878 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:17.559832 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391f943e-13e1-43e9-b6be-da08145df094 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:17.621474 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54ea798-f5dc-43d7-9ed4-a706f8916c78 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:17.655231 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad47fdb-62aa-4727-b681-350454c40821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:17.656161 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad47fdb-62aa-4727-b681-350454c40821 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:17.656571 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad47fdb-62aa-4727-b681-350454c40821 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:17.656972 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad47fdb-62aa-4727-b681-350454c40821 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:17.679502 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64627bd3-5c3e-4212-953c-3b3afca8d93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:17.680397 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64627bd3-5c3e-4212-953c-3b3afca8d93b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:17.680824 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64627bd3-5c3e-4212-953c-3b3afca8d93b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:17.681236 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64627bd3-5c3e-4212-953c-3b3afca8d93b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:17.696732 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64627bd3-5c3e-4212-953c-3b3afca8d93b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:17.702911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-64627bd3-5c3e-4212-953c-3b3afca8d93b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 15:28:17.702911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:17.702911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:17.702911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:17.702911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:17.702911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:17.702911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:17.702911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:17.702911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:17.702911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:17.702911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:17.702911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:17.702911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:17.702911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:17.702911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:17.702911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:17.703990 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:17.703990 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:17.703990 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:17.703990 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:17.703990 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:17.703990 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:17.703990 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:17.703990 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:17.703990 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:17.703990 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:17.703990 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:17.703990 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:17.703990 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:17.703990 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:17.703990 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:17.703990 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:17.704954 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:17.704954 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:17.704954 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:17.704954 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:17.704954 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:17.704954 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:17.704954 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:17.704954 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:17.704954 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:17.704954 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:17.704954 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 28 15:28:17.704954 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:17.711374 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 176/343] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:28:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:28:17.717915 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64627bd3-5c3e-4212-953c-3b3afca8d93b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:17.720814 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:17.760363 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad47fdb-62aa-4727-b681-350454c40821 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:17.761830 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad47fdb-62aa-4727-b681-350454c40821 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:17.842560 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:17.864715 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64627bd3-5c3e-4212-953c-3b3afca8d93b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:17.882644 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 168/344] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:28:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:28:17.887933 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad47fdb-62aa-4727-b681-350454c40821 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:17.917898 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19033af4-d753-4bb5-8a84-07afe604e315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:17.918809 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19033af4-d753-4bb5-8a84-07afe604e315 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:17.919228 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19033af4-d753-4bb5-8a84-07afe604e315 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:17.919640 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19033af4-d753-4bb5-8a84-07afe604e315 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:17.920441 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19033af4-d753-4bb5-8a84-07afe604e315 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:17.920882 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19033af4-d753-4bb5-8a84-07afe604e315 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:17.922438 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19033af4-d753-4bb5-8a84-07afe604e315 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:17.935954 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 177/345] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 15:28:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:28:17.936981 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19033af4-d753-4bb5-8a84-07afe604e315 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:17.963951 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:17.997778 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad47fdb-62aa-4727-b681-350454c40821 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:18.035549 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b188701-0e15-4e30-bfa0-f17dcf43fd3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:18.036185 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b188701-0e15-4e30-bfa0-f17dcf43fd3b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:18.036455 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b188701-0e15-4e30-bfa0-f17dcf43fd3b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:18.036721 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b188701-0e15-4e30-bfa0-f17dcf43fd3b None admin] PATH_INFO: `/v3/projects/4251d157536c4934bf6f66268f3037a9` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:18.037960 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b188701-0e15-4e30-bfa0-f17dcf43fd3b None admin] RBAC: Authorizing `identity:get_project(project_id=4251d157536c4934bf6f66268f3037a9)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:18.039508 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b188701-0e15-4e30-bfa0-f17dcf43fd3b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:18.042124 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 169/346] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:28:17 2022] GET /identity/v3/projects/4251d157536c4934bf6f66268f3037a9 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:28:18.042923 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b188701-0e15-4e30-bfa0-f17dcf43fd3b None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:18.062468 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19033af4-d753-4bb5-8a84-07afe604e315 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:18.101249 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889e1a4b-9567-40b7-9be0-6907c25b9400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:18.101805 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e1a4b-9567-40b7-9be0-6907c25b9400 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:18.102149 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e1a4b-9567-40b7-9be0-6907c25b9400 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:18.102404 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e1a4b-9567-40b7-9be0-6907c25b9400 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:18.114502 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889e1a4b-9567-40b7-9be0-6907c25b9400 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:18.120191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-889e1a4b-9567-40b7-9be0-6907c25b9400 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:28:18.120191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:18.120191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:18.120191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:18.120191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:18.120191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:18.120191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:18.120191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:18.120191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:18.120191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:18.120191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:18.120191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:18.120191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:18.120191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:18.120191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:18.120191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:18.121191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:18.121191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:18.121191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:18.121191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:18.121191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:18.121191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:18.121191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:18.121191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:18.121191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:18.121191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:18.121191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:18.121191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:18.121191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:18.121191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:18.121191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:18.121191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:18.122043 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:18.122043 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:18.122043 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:18.122043 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:18.122043 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:18.122043 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:18.122043 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:18.122043 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:18.122043 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:18.122043 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:18.122043 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:28:18.122043 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:18.122043 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 178/347] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 15:28:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:18.123196 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75324599-85fd-4d68-8d1c-f782f4273a40 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:18.123196 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75324599-85fd-4d68-8d1c-f782f4273a40 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:18.123476 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75324599-85fd-4d68-8d1c-f782f4273a40 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:18.128825 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 179/348] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:18.128825 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75324599-85fd-4d68-8d1c-f782f4273a40 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:18.128825 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:18.160627 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b188701-0e15-4e30-bfa0-f17dcf43fd3b None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:18.196854 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5587c3e7-8aac-4bae-b0f7-b306b52d9aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:18.197536 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5587c3e7-8aac-4bae-b0f7-b306b52d9aea None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:18.197810 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5587c3e7-8aac-4bae-b0f7-b306b52d9aea None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:18.198117 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5587c3e7-8aac-4bae-b0f7-b306b52d9aea None admin] PATH_INFO: `/v3/users/8137d773fc2c46bd896204082fc8e81c` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:18.199227 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5587c3e7-8aac-4bae-b0f7-b306b52d9aea None admin] RBAC: Authorizing `identity:get_user(user_id=8137d773fc2c46bd896204082fc8e81c)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:18.200661 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5587c3e7-8aac-4bae-b0f7-b306b52d9aea None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:18.202552 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 170/349] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:28:18 2022] GET /identity/v3/users/8137d773fc2c46bd896204082fc8e81c => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:18.204574 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b3597-cae7-4757-9159-15acdfe39bc2 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:18.204850 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b3597-cae7-4757-9159-15acdfe39bc2 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:18.205118 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b3597-cae7-4757-9159-15acdfe39bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:18.223846 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-684b3597-cae7-4757-9159-15acdfe39bc2 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:18.225039 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-684b3597-cae7-4757-9159-15acdfe39bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:18.227192 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:18.247598 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75324599-85fd-4d68-8d1c-f782f4273a40 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:18.297698 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2f4d6f-e12e-46fa-8056-b2383b9ca003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:18.298690 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2f4d6f-e12e-46fa-8056-b2383b9ca003 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:18.298690 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2f4d6f-e12e-46fa-8056-b2383b9ca003 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:18.298988 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2f4d6f-e12e-46fa-8056-b2383b9ca003 None admin] PATH_INFO: `/v3/groups/0109f2c831fd421582207689515d2d91` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:18.304164 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 171/350] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:18.305803 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684b3597-cae7-4757-9159-15acdfe39bc2 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:18.307710 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2f4d6f-e12e-46fa-8056-b2383b9ca003 None admin] RBAC: Authorizing `identity:get_group(group_id=0109f2c831fd421582207689515d2d91)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:18.309257 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2f4d6f-e12e-46fa-8056-b2383b9ca003 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:18.312395 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 180/351] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Sep 28 15:28:18 2022] GET /identity/v3/groups/0109f2c831fd421582207689515d2d91 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:18.313196 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2f4d6f-e12e-46fa-8056-b2383b9ca003 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:18.439262 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:18.487683 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684b3597-cae7-4757-9159-15acdfe39bc2 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:18.503933 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2f4d6f-e12e-46fa-8056-b2383b9ca003 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:18.549235 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b44b710-8c45-4e5b-87ec-fcf134e6876b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:18.549877 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44b710-8c45-4e5b-87ec-fcf134e6876b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:18.550138 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44b710-8c45-4e5b-87ec-fcf134e6876b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:18.550401 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44b710-8c45-4e5b-87ec-fcf134e6876b None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:18.551288 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b44b710-8c45-4e5b-87ec-fcf134e6876b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:18.551583 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b44b710-8c45-4e5b-87ec-fcf134e6876b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:18.552941 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b44b710-8c45-4e5b-87ec-fcf134e6876b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:18.564700 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8008dfe1-a1ba-41f3-a3ad-acef4c1c4b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:18.565314 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8008dfe1-a1ba-41f3-a3ad-acef4c1c4b86 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:18.565586 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8008dfe1-a1ba-41f3-a3ad-acef4c1c4b86 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:18.565865 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8008dfe1-a1ba-41f3-a3ad-acef4c1c4b86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:18.566585 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8008dfe1-a1ba-41f3-a3ad-acef4c1c4b86 None admin] RBAC: Adding query filter params (role.id=9e4cabcdb1e349c699da538ce00d1396, user.id=8137d773fc2c46bd896204082fc8e81c, scope.system=all) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:18.566887 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8008dfe1-a1ba-41f3-a3ad-acef4c1c4b86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:18.568325 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8008dfe1-a1ba-41f3-a3ad-acef4c1c4b86 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:18.570427 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:18.573312 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 172/352] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:28:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:18.574149 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b44b710-8c45-4e5b-87ec-fcf134e6876b None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:18.595271 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 181/353] 10.222.0.52 () {56 vars in 1308 bytes} [Wed Sep 28 15:28:18 2022] GET /identity/v3/role_assignments?user.id=8137d773fc2c46bd896204082fc8e81c&scope.system=all&role.id=9e4cabcdb1e349c699da538ce00d1396 => generated 510 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:18.596124 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8008dfe1-a1ba-41f3-a3ad-acef4c1c4b86 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:18.729720 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b44b710-8c45-4e5b-87ec-fcf134e6876b None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:18.769994 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa87de02-ccdc-4f9d-bb1c-74943c20c31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:18.770623 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87de02-ccdc-4f9d-bb1c-74943c20c31d None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:18.770895 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87de02-ccdc-4f9d-bb1c-74943c20c31d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:18.771162 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87de02-ccdc-4f9d-bb1c-74943c20c31d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:18.771899 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa87de02-ccdc-4f9d-bb1c-74943c20c31d None admin] RBAC: Adding query filter params (group.id=0109f2c831fd421582207689515d2d91, role.id=e152a36b5a1b470b912b1d705462f512, scope.project.id=4251d157536c4934bf6f66268f3037a9) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:18.772206 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa87de02-ccdc-4f9d-bb1c-74943c20c31d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:18.773655 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa87de02-ccdc-4f9d-bb1c-74943c20c31d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:18.780361 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 173/354] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Sep 28 15:28:18 2022] GET /identity/v3/role_assignments?group.id=0109f2c831fd421582207689515d2d91&scope.project.id=4251d157536c4934bf6f66268f3037a9&role.id=e152a36b5a1b470b912b1d705462f512 => generated 260 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:18.781305 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa87de02-ccdc-4f9d-bb1c-74943c20c31d None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:18.786589 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8008dfe1-a1ba-41f3-a3ad-acef4c1c4b86 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:18.813206 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:18.827634 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a27e6d1-4636-4414-aa49-403d3d781ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:18.828447 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a27e6d1-4636-4414-aa49-403d3d781ddc None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:18.828796 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a27e6d1-4636-4414-aa49-403d3d781ddc None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:18.829078 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a27e6d1-4636-4414-aa49-403d3d781ddc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:18.886931 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:18.910705 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a27e6d1-4636-4414-aa49-403d3d781ddc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:18.912430 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a27e6d1-4636-4414-aa49-403d3d781ddc None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:18.936455 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa87de02-ccdc-4f9d-bb1c-74943c20c31d None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:18.976790 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f35fb08-55a8-4347-a5f3-da4006dddc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:18.977723 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f35fb08-55a8-4347-a5f3-da4006dddc51 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:18.978199 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f35fb08-55a8-4347-a5f3-da4006dddc51 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:18.979379 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f35fb08-55a8-4347-a5f3-da4006dddc51 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:18.992602 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f35fb08-55a8-4347-a5f3-da4006dddc51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:18.998512 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-0f35fb08-55a8-4347-a5f3-da4006dddc51 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:28:18.998512 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:18.998512 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:18.998512 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:18.998512 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:18.998512 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:18.998512 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:18.998512 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:18.998512 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:18.998512 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:18.998512 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:18.998512 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:18.998512 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:18.998512 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:18.998512 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:18.998512 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:18.999444 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:18.999444 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:18.999444 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:18.999444 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:18.999444 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:18.999444 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:18.999444 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:18.999444 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:18.999444 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:18.999444 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:18.999444 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:18.999444 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:18.999444 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:18.999444 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:18.999444 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:18.999444 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:19.000237 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:19.000237 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:19.000237 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:19.000237 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:19.000237 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:19.000237 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:19.000237 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:19.000237 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:19.000237 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:19.000237 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:19.000237 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:28:19.000237 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:19.002637 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 174/355] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:28:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:19.004151 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca561f6-0ce3-47b1-9b79-4cb99c7d8173 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:19.004589 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca561f6-0ce3-47b1-9b79-4cb99c7d8173 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:19.005054 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca561f6-0ce3-47b1-9b79-4cb99c7d8173 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:19.007080 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 175/356] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:19.012587 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200208b9-cee4-4132-b5e0-f353178fc18c None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:19.012866 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200208b9-cee4-4132-b5e0-f353178fc18c None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:19.013119 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200208b9-cee4-4132-b5e0-f353178fc18c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:19.034899 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 182/357] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:28:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:28:19.035087 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-200208b9-cee4-4132-b5e0-f353178fc18c None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:19.036158 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-200208b9-cee4-4132-b5e0-f353178fc18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:19.036369 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a27e6d1-4636-4414-aa49-403d3d781ddc None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:19.075194 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:19.137728 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 176/358] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:19.138913 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200208b9-cee4-4132-b5e0-f353178fc18c None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:19.171566 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a27e6d1-4636-4414-aa49-403d3d781ddc None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:19.212795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2591cfcc-7c41-4921-9e72-b0f94085a62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:19.213464 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:19.214066 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591cfcc-7c41-4921-9e72-b0f94085a62d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:19.214543 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591cfcc-7c41-4921-9e72-b0f94085a62d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:19.214995 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2591cfcc-7c41-4921-9e72-b0f94085a62d None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:19.216236 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2591cfcc-7c41-4921-9e72-b0f94085a62d None admin] RBAC: Adding query filter params (name=member) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:19.216761 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2591cfcc-7c41-4921-9e72-b0f94085a62d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:19.218373 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2591cfcc-7c41-4921-9e72-b0f94085a62d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:19.227839 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 183/359] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 15:28:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:19.235215 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2591cfcc-7c41-4921-9e72-b0f94085a62d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:19.267888 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200208b9-cee4-4132-b5e0-f353178fc18c None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:19.333135 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a06ae1d-2769-46e4-a1f8-37281597a2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:19.333679 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a06ae1d-2769-46e4-a1f8-37281597a2b0 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:19.333984 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a06ae1d-2769-46e4-a1f8-37281597a2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:19.334258 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a06ae1d-2769-46e4-a1f8-37281597a2b0 None admin] PATH_INFO: `/v3/projects/4251d157536c4934bf6f66268f3037a9` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:19.335433 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a06ae1d-2769-46e4-a1f8-37281597a2b0 None admin] RBAC: Authorizing `identity:get_project(project_id=4251d157536c4934bf6f66268f3037a9)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:19.336706 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:19.336908 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a06ae1d-2769-46e4-a1f8-37281597a2b0 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:19.338996 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 177/360] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:28:19 2022] GET /identity/v3/projects/4251d157536c4934bf6f66268f3037a9 => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:19.346816 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a06ae1d-2769-46e4-a1f8-37281597a2b0 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:19.354524 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2591cfcc-7c41-4921-9e72-b0f94085a62d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:19.393730 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac54349c-c787-4e17-82bc-d82cbfe82e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:19.394700 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54349c-c787-4e17-82bc-d82cbfe82e21 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:19.395173 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54349c-c787-4e17-82bc-d82cbfe82e21 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:19.395621 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54349c-c787-4e17-82bc-d82cbfe82e21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:19.482002 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac54349c-c787-4e17-82bc-d82cbfe82e21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:19.483838 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac54349c-c787-4e17-82bc-d82cbfe82e21 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:19.516078 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a06ae1d-2769-46e4-a1f8-37281597a2b0 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:19.549158 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:19.572502 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d136d92b-ccd3-4ad0-860c-4ee9f4915e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:19.573649 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d136d92b-ccd3-4ad0-860c-4ee9f4915e7c None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:19.574120 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d136d92b-ccd3-4ad0-860c-4ee9f4915e7c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:19.574702 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d136d92b-ccd3-4ad0-860c-4ee9f4915e7c None admin] PATH_INFO: `/v3/groups/0109f2c831fd421582207689515d2d91` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:19.576730 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d136d92b-ccd3-4ad0-860c-4ee9f4915e7c None admin] RBAC: Authorizing `identity:get_group(group_id=0109f2c831fd421582207689515d2d91)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:19.578387 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d136d92b-ccd3-4ad0-860c-4ee9f4915e7c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:19.580403 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 178/361] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Sep 28 15:28:19 2022] GET /identity/v3/groups/0109f2c831fd421582207689515d2d91 => generated 226 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:19.582173 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b8e55-8300-4ca5-b579-92e1a7bb2aa4 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:19.582618 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b8e55-8300-4ca5-b579-92e1a7bb2aa4 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:19.583415 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401b8e55-8300-4ca5-b579-92e1a7bb2aa4 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:19.585587 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 179/362] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:19.586533 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b8e55-8300-4ca5-b579-92e1a7bb2aa4 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:19.592191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:19.615712 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 184/363] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:28:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 28 15:28:19.615917 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac54349c-c787-4e17-82bc-d82cbfe82e21 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:19.741064 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401b8e55-8300-4ca5-b579-92e1a7bb2aa4 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:19.763028 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac54349c-c787-4e17-82bc-d82cbfe82e21 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:19.783283 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03de990-9898-4720-9461-46628cac57d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:19.783928 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03de990-9898-4720-9461-46628cac57d7 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:19.784200 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03de990-9898-4720-9461-46628cac57d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:19.784469 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03de990-9898-4720-9461-46628cac57d7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:19.803195 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03de990-9898-4720-9461-46628cac57d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:19.809975 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-d03de990-9898-4720-9461-46628cac57d7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:28:19.809975 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:19.809975 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:19.809975 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:19.809975 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:19.809975 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:19.809975 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:19.809975 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:19.809975 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:19.809975 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:19.809975 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:19.809975 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:19.809975 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:19.809975 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:19.809975 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:19.809975 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:19.810981 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:19.810981 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:19.810981 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:19.810981 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:19.810981 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:19.810981 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:19.810981 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:19.810981 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:19.810981 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:19.810981 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:19.810981 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:19.810981 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:19.810981 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:19.810981 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:19.810981 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:19.810981 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:19.812029 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:19.812029 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:19.812029 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:19.812029 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:19.812029 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:19.812029 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:19.812029 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:19.812029 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:19.812029 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:19.812029 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:19.812029 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:28:19.812029 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:19.812029 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 180/364] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 15:28:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:19.812860 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c560e76-3109-4953-8c9d-215dcd7dcb51 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:19.812860 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c560e76-3109-4953-8c9d-215dcd7dcb51 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:19.812860 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c560e76-3109-4953-8c9d-215dcd7dcb51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:19.823191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e09c4b-6a95-444a-9be2-37e0be1a294a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:19.824036 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e09c4b-6a95-444a-9be2-37e0be1a294a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:19.824463 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e09c4b-6a95-444a-9be2-37e0be1a294a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:19.824923 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e09c4b-6a95-444a-9be2-37e0be1a294a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:19.825799 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e09c4b-6a95-444a-9be2-37e0be1a294a None admin] RBAC: Adding query filter params (group.id=0109f2c831fd421582207689515d2d91, role.id=5038c8140b7145ceb7494210a5c9f9f0, scope.project.id=4251d157536c4934bf6f66268f3037a9) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:19.826298 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e09c4b-6a95-444a-9be2-37e0be1a294a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:19.827992 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e09c4b-6a95-444a-9be2-37e0be1a294a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:19.841378 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-0c560e76-3109-4953-8c9d-215dcd7dcb51 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:19.843838 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-0c560e76-3109-4953-8c9d-215dcd7dcb51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:19.844359 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 185/365] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Sep 28 15:28:19 2022] GET /identity/v3/role_assignments?group.id=0109f2c831fd421582207689515d2d91&scope.project.id=4251d157536c4934bf6f66268f3037a9&role.id=5038c8140b7145ceb7494210a5c9f9f0 => generated 260 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:19.845922 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e09c4b-6a95-444a-9be2-37e0be1a294a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:19.926313 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:19.944737 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:19.977134 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 181/366] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:19.982174 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c560e76-3109-4953-8c9d-215dcd7dcb51 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:20.033160 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e09c4b-6a95-444a-9be2-37e0be1a294a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:20.121546 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf9d64b-77db-4d7a-85f4-273718655f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:20.124382 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9d64b-77db-4d7a-85f4-273718655f30 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:20.125221 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9d64b-77db-4d7a-85f4-273718655f30 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:20.125687 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9d64b-77db-4d7a-85f4-273718655f30 None admin] PATH_INFO: `/v3/projects/fbb3348b420b4f6ba5d171c387e5a403` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:20.128484 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf9d64b-77db-4d7a-85f4-273718655f30 None admin] RBAC: Authorizing `identity:get_project(project_id=fbb3348b420b4f6ba5d171c387e5a403)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:20.130324 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf9d64b-77db-4d7a-85f4-273718655f30 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:20.133759 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 186/367] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:28:19 2022] GET /identity/v3/projects/fbb3348b420b4f6ba5d171c387e5a403 => generated 302 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:20.143124 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf9d64b-77db-4d7a-85f4-273718655f30 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:20.265763 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c560e76-3109-4953-8c9d-215dcd7dcb51 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:20.320615 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf9d64b-77db-4d7a-85f4-273718655f30 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:20.325349 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc4a61e-4265-40c5-a2ad-f06a8de83c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:20.326606 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc4a61e-4265-40c5-a2ad-f06a8de83c10 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:20.326606 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc4a61e-4265-40c5-a2ad-f06a8de83c10 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:20.326606 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc4a61e-4265-40c5-a2ad-f06a8de83c10 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:20.327271 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc4a61e-4265-40c5-a2ad-f06a8de83c10 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:20.327567 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc4a61e-4265-40c5-a2ad-f06a8de83c10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:20.328923 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc4a61e-4265-40c5-a2ad-f06a8de83c10 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:20.345104 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 182/368] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:28:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:20.346155 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:20.347200 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc4a61e-4265-40c5-a2ad-f06a8de83c10 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:20.378963 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c106e2cc-c423-4ffe-bb17-822a15eac5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:20.379722 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c106e2cc-c423-4ffe-bb17-822a15eac5e6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:20.380100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c106e2cc-c423-4ffe-bb17-822a15eac5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:20.380380 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c106e2cc-c423-4ffe-bb17-822a15eac5e6 None admin] PATH_INFO: `/v3/groups/0109f2c831fd421582207689515d2d91` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:20.384065 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c106e2cc-c423-4ffe-bb17-822a15eac5e6 None admin] RBAC: Authorizing `identity:get_group(group_id=0109f2c831fd421582207689515d2d91)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:20.386277 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c106e2cc-c423-4ffe-bb17-822a15eac5e6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:20.388037 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 187/369] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Sep 28 15:28:20 2022] GET /identity/v3/groups/0109f2c831fd421582207689515d2d91 => generated 226 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:20.388873 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c106e2cc-c423-4ffe-bb17-822a15eac5e6 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:20.400018 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:20.460291 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc4a61e-4265-40c5-a2ad-f06a8de83c10 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:20.506964 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebcb9bf-6977-4f13-bae3-630d9a9ff568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:20.508226 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcb9bf-6977-4f13-bae3-630d9a9ff568 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:20.508917 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcb9bf-6977-4f13-bae3-630d9a9ff568 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:20.509380 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcb9bf-6977-4f13-bae3-630d9a9ff568 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:20.516054 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c106e2cc-c423-4ffe-bb17-822a15eac5e6 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:20.551812 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed8dd45-87a8-46d9-95c7-da97580a45d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:20.552500 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8dd45-87a8-46d9-95c7-da97580a45d7 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:20.552789 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8dd45-87a8-46d9-95c7-da97580a45d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:20.553065 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8dd45-87a8-46d9-95c7-da97580a45d7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:20.566306 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed8dd45-87a8-46d9-95c7-da97580a45d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:20.572071 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-2ed8dd45-87a8-46d9-95c7-da97580a45d7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:28:20.572071 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:20.572071 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:20.572071 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:20.572071 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:20.572071 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:20.572071 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:20.572071 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:20.572071 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:20.572071 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:20.572071 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:20.572071 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:20.572071 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:20.572071 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:20.572071 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:20.572071 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:20.573606 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:20.573606 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:20.573606 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:20.573606 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:20.573606 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:20.573606 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:20.573606 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:20.573606 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:20.573606 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:20.573606 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:20.573606 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:20.573606 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:20.573606 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:20.573606 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:20.573606 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:20.573606 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:20.574446 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:20.574446 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:20.574446 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:20.574446 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:20.574446 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:20.574446 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:20.574446 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:20.574446 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:20.574446 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:20.574446 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:20.574446 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:28:20.574446 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:20.574446 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 188/370] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 15:28:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:20.575104 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed8dd45-87a8-46d9-95c7-da97580a45d7 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:20.588200 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebcb9bf-6977-4f13-bae3-630d9a9ff568 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:20.590396 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebcb9bf-6977-4f13-bae3-630d9a9ff568 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:20.631508 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:20.638618 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:20.694893 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed8dd45-87a8-46d9-95c7-da97580a45d7 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:20.699039 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 183/371] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:28:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:28:20.699965 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebcb9bf-6977-4f13-bae3-630d9a9ff568 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:20.734060 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c292e8f5-9927-4a23-a7ac-5f699db7ab9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:20.734976 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292e8f5-9927-4a23-a7ac-5f699db7ab9f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:20.735446 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292e8f5-9927-4a23-a7ac-5f699db7ab9f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:20.735902 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292e8f5-9927-4a23-a7ac-5f699db7ab9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:20.737035 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c292e8f5-9927-4a23-a7ac-5f699db7ab9f None admin] RBAC: Adding query filter params (group.id=0109f2c831fd421582207689515d2d91, role.id=e152a36b5a1b470b912b1d705462f512, scope.project.id=fbb3348b420b4f6ba5d171c387e5a403) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:20.737624 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c292e8f5-9927-4a23-a7ac-5f699db7ab9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:20.739315 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c292e8f5-9927-4a23-a7ac-5f699db7ab9f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:20.746304 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 189/372] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Sep 28 15:28:20 2022] GET /identity/v3/role_assignments?group.id=0109f2c831fd421582207689515d2d91&scope.project.id=fbb3348b420b4f6ba5d171c387e5a403&role.id=e152a36b5a1b470b912b1d705462f512 => generated 260 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:20.747323 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c292e8f5-9927-4a23-a7ac-5f699db7ab9f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:20.813662 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebcb9bf-6977-4f13-bae3-630d9a9ff568 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:20.851751 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbfb6c6-17e8-4375-a645-772d25024a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:20.852917 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbfb6c6-17e8-4375-a645-772d25024a90 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:20.854449 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbfb6c6-17e8-4375-a645-772d25024a90 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:20.854592 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbfb6c6-17e8-4375-a645-772d25024a90 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:20.855776 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbfb6c6-17e8-4375-a645-772d25024a90 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:20.856327 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbfb6c6-17e8-4375-a645-772d25024a90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:20.858304 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbfb6c6-17e8-4375-a645-772d25024a90 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:20.869896 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 184/373] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:28:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:20.872693 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c292e8f5-9927-4a23-a7ac-5f699db7ab9f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:20.878227 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbfb6c6-17e8-4375-a645-772d25024a90 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:20.882799 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:20.887732 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:20.917653 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d345a2f-705c-4758-858a-6a38f75bbd0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:20.918692 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d345a2f-705c-4758-858a-6a38f75bbd0e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:20.919177 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d345a2f-705c-4758-858a-6a38f75bbd0e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:20.919671 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d345a2f-705c-4758-858a-6a38f75bbd0e None admin] PATH_INFO: `/v3/projects/fbb3348b420b4f6ba5d171c387e5a403` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:20.921110 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d345a2f-705c-4758-858a-6a38f75bbd0e None admin] RBAC: Authorizing `identity:get_project(project_id=fbb3348b420b4f6ba5d171c387e5a403)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:20.922810 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d345a2f-705c-4758-858a-6a38f75bbd0e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:20.924811 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 190/374] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:28:20 2022] GET /identity/v3/projects/fbb3348b420b4f6ba5d171c387e5a403 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:20.930801 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d345a2f-705c-4758-858a-6a38f75bbd0e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:20.990790 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbfb6c6-17e8-4375-a645-772d25024a90 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:21.031959 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16e9af8-1a57-42db-b580-fd198c2fff09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:21.032581 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e9af8-1a57-42db-b580-fd198c2fff09 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:21.032856 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e9af8-1a57-42db-b580-fd198c2fff09 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:21.033123 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16e9af8-1a57-42db-b580-fd198c2fff09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:21.053267 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d345a2f-705c-4758-858a-6a38f75bbd0e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:21.092221 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43700d91-fb73-48ad-af66-8725c256a240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:21.093263 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43700d91-fb73-48ad-af66-8725c256a240 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:21.093746 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43700d91-fb73-48ad-af66-8725c256a240 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:21.094272 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43700d91-fb73-48ad-af66-8725c256a240 None admin] PATH_INFO: `/v3/groups/0109f2c831fd421582207689515d2d91` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:21.095900 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43700d91-fb73-48ad-af66-8725c256a240 None admin] RBAC: Authorizing `identity:get_group(group_id=0109f2c831fd421582207689515d2d91)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:21.097718 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43700d91-fb73-48ad-af66-8725c256a240 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:21.099945 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 191/375] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Sep 28 15:28:20 2022] GET /identity/v3/groups/0109f2c831fd421582207689515d2d91 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:21.106505 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43700d91-fb73-48ad-af66-8725c256a240 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:21.109448 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16e9af8-1a57-42db-b580-fd198c2fff09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:21.111687 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16e9af8-1a57-42db-b580-fd198c2fff09 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:21.113796 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:21.164348 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:21.226883 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43700d91-fb73-48ad-af66-8725c256a240 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:21.251415 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 185/376] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:28:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:28:21.257494 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16e9af8-1a57-42db-b580-fd198c2fff09 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:21.266000 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e7800b-bcef-4fc1-9d0c-b417bbcf8f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:21.266935 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7800b-bcef-4fc1-9d0c-b417bbcf8f91 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:21.267092 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7800b-bcef-4fc1-9d0c-b417bbcf8f91 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:21.267460 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7800b-bcef-4fc1-9d0c-b417bbcf8f91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:21.268330 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e7800b-bcef-4fc1-9d0c-b417bbcf8f91 None admin] RBAC: Adding query filter params (group.id=0109f2c831fd421582207689515d2d91, role.id=5038c8140b7145ceb7494210a5c9f9f0, scope.project.id=fbb3348b420b4f6ba5d171c387e5a403) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:21.268730 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e7800b-bcef-4fc1-9d0c-b417bbcf8f91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:21.272936 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e7800b-bcef-4fc1-9d0c-b417bbcf8f91 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:21.279275 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 192/377] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Sep 28 15:28:21 2022] GET /identity/v3/role_assignments?group.id=0109f2c831fd421582207689515d2d91&scope.project.id=fbb3348b420b4f6ba5d171c387e5a403&role.id=5038c8140b7145ceb7494210a5c9f9f0 => generated 260 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:28:21.391452 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16e9af8-1a57-42db-b580-fd198c2fff09 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:21.395212 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:21.435848 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a84d9-f5c0-430b-b2db-1617050e42da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:21.436412 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a84d9-f5c0-430b-b2db-1617050e42da None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:21.436688 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a84d9-f5c0-430b-b2db-1617050e42da None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:21.436959 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a84d9-f5c0-430b-b2db-1617050e42da None admin] PATH_INFO: `/v3/projects/8de950cb25d34fe8afed2528d2e112e6` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:21.438279 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656a84d9-f5c0-430b-b2db-1617050e42da None admin] RBAC: Authorizing `identity:get_project(project_id=8de950cb25d34fe8afed2528d2e112e6)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:21.439760 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656a84d9-f5c0-430b-b2db-1617050e42da None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:21.441949 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 186/378] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:28:21 2022] GET /identity/v3/projects/8de950cb25d34fe8afed2528d2e112e6 => generated 344 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:28:21.448208 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e7800b-bcef-4fc1-9d0c-b417bbcf8f91 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:21.578226 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e7800b-bcef-4fc1-9d0c-b417bbcf8f91 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:21.591450 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:21.611030 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a4e11f-cb45-4b19-b6bc-76222512eabf None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:21.611560 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a4e11f-cb45-4b19-b6bc-76222512eabf None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:21.611560 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a4e11f-cb45-4b19-b6bc-76222512eabf None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:21.612860 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 187/379] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:28:21.623637 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631b26ae-12af-475d-a1a1-b4154d5787c4 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:21.623929 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631b26ae-12af-475d-a1a1-b4154d5787c4 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:21.624183 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631b26ae-12af-475d-a1a1-b4154d5787c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:21.625353 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538f12d0-f9ef-4620-94bd-a1eea4af6902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:21.626004 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f12d0-f9ef-4620-94bd-a1eea4af6902 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:21.626279 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f12d0-f9ef-4620-94bd-a1eea4af6902 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:21.626549 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f12d0-f9ef-4620-94bd-a1eea4af6902 None admin] PATH_INFO: `/v3/groups/ac94b5bc6e804e1db1bf63b9390ad35f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:21.641471 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538f12d0-f9ef-4620-94bd-a1eea4af6902 None admin] RBAC: Authorizing `identity:get_group(group_id=ac94b5bc6e804e1db1bf63b9390ad35f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:21.643795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538f12d0-f9ef-4620-94bd-a1eea4af6902 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:21.649874 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 193/380] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Sep 28 15:28:21 2022] GET /identity/v3/groups/ac94b5bc6e804e1db1bf63b9390ad35f => generated 229 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:21.650040 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-631b26ae-12af-475d-a1a1-b4154d5787c4 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:21.650040 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-631b26ae-12af-475d-a1a1-b4154d5787c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:21.652243 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538f12d0-f9ef-4620-94bd-a1eea4af6902 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:21.799091 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 188/381] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:28:21.844652 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538f12d0-f9ef-4620-94bd-a1eea4af6902 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:21.885933 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38b5222-cc5a-46f0-9f2b-1ad921697f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:21.886557 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b5222-cc5a-46f0-9f2b-1ad921697f3e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:21.886838 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b5222-cc5a-46f0-9f2b-1ad921697f3e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:21.887112 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b5222-cc5a-46f0-9f2b-1ad921697f3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:21.887838 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38b5222-cc5a-46f0-9f2b-1ad921697f3e None admin] RBAC: Adding query filter params (group.id=ac94b5bc6e804e1db1bf63b9390ad35f, role.id=62d96ac9513c452894fc67b442cfc59f, scope.project.id=8de950cb25d34fe8afed2528d2e112e6) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:21.888144 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38b5222-cc5a-46f0-9f2b-1ad921697f3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:21.889763 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38b5222-cc5a-46f0-9f2b-1ad921697f3e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:21.896578 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 194/382] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Sep 28 15:28:21 2022] GET /identity/v3/role_assignments?group.id=ac94b5bc6e804e1db1bf63b9390ad35f&scope.project.id=8de950cb25d34fe8afed2528d2e112e6&role.id=62d96ac9513c452894fc67b442cfc59f => generated 260 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:28:22.146614 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631b26ae-12af-475d-a1a1-b4154d5787c4 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:22.269478 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631b26ae-12af-475d-a1a1-b4154d5787c4 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:22.271176 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:22.307856 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9803bb-44f1-4c02-a909-3ff582edf9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:22.308490 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9803bb-44f1-4c02-a909-3ff582edf9fb None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:22.308765 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9803bb-44f1-4c02-a909-3ff582edf9fb None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:22.309034 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9803bb-44f1-4c02-a909-3ff582edf9fb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:22.319692 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9803bb-44f1-4c02-a909-3ff582edf9fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:22.325098 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-fc9803bb-44f1-4c02-a909-3ff582edf9fb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:28:22.325098 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:22.325098 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:22.325098 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:22.325098 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:22.325098 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:22.325098 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:22.325098 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:22.325098 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:22.325098 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:22.325098 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:22.325098 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:22.325098 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:22.325098 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:22.325098 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:22.325098 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:22.326082 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:22.326082 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:22.326082 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:22.326082 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:22.326082 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:22.326082 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:22.326082 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:22.326082 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:22.326082 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:22.326082 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:22.326082 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:22.326082 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:22.326082 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:22.326082 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:22.326082 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:22.326082 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:22.326878 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:22.326878 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:22.326878 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:22.326878 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:22.326878 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:22.326878 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:22.326878 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:22.326878 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:22.326878 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:22.326878 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:22.326878 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:28:22.326878 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:22.326878 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 189/383] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:28:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:22.331944 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38b5222-cc5a-46f0-9f2b-1ad921697f3e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:22.375446 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:22.439204 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38b5222-cc5a-46f0-9f2b-1ad921697f3e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:22.477354 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63de3bf6-ebe3-4171-94e9-44e4d8925517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:22.477991 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de3bf6-ebe3-4171-94e9-44e4d8925517 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:22.478276 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de3bf6-ebe3-4171-94e9-44e4d8925517 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:22.479067 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de3bf6-ebe3-4171-94e9-44e4d8925517 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:22.479725 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63de3bf6-ebe3-4171-94e9-44e4d8925517 None admin] RBAC: Adding query filter params (name=member) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:22.480028 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63de3bf6-ebe3-4171-94e9-44e4d8925517 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:22.481411 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63de3bf6-ebe3-4171-94e9-44e4d8925517 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:22.490279 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 195/384] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 15:28:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:28:22.496527 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9803bb-44f1-4c02-a909-3ff582edf9fb None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:22.604472 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9803bb-44f1-4c02-a909-3ff582edf9fb None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:22.640214 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13edbe67-6d1c-45a0-b277-341ed3a2dd73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:22.640827 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13edbe67-6d1c-45a0-b277-341ed3a2dd73 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:22.641099 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13edbe67-6d1c-45a0-b277-341ed3a2dd73 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:22.641368 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13edbe67-6d1c-45a0-b277-341ed3a2dd73 None admin] PATH_INFO: `/v3/groups/0109f2c831fd421582207689515d2d91` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:22.642491 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13edbe67-6d1c-45a0-b277-341ed3a2dd73 None admin] RBAC: Authorizing `identity:get_group(group_id=0109f2c831fd421582207689515d2d91)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:22.643932 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13edbe67-6d1c-45a0-b277-341ed3a2dd73 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:22.645833 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 190/385] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Sep 28 15:28:22 2022] GET /identity/v3/groups/0109f2c831fd421582207689515d2d91 => generated 226 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:22.651494 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63de3bf6-ebe3-4171-94e9-44e4d8925517 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:22.786104 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:22.790585 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63de3bf6-ebe3-4171-94e9-44e4d8925517 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:22.833986 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6ae5bc-ec38-481c-ac92-c7777c88cbd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:22.834625 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ae5bc-ec38-481c-ac92-c7777c88cbd4 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:22.834907 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ae5bc-ec38-481c-ac92-c7777c88cbd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:22.835179 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ae5bc-ec38-481c-ac92-c7777c88cbd4 None admin] PATH_INFO: `/v3/projects/4251d157536c4934bf6f66268f3037a9` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:22.836309 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6ae5bc-ec38-481c-ac92-c7777c88cbd4 None admin] RBAC: Authorizing `identity:get_project(project_id=4251d157536c4934bf6f66268f3037a9)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:22.838324 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6ae5bc-ec38-481c-ac92-c7777c88cbd4 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:22.840294 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 196/386] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:28:22 2022] GET /identity/v3/projects/4251d157536c4934bf6f66268f3037a9 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:22.845988 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13edbe67-6d1c-45a0-b277-341ed3a2dd73 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:22.891059 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:22.976859 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13edbe67-6d1c-45a0-b277-341ed3a2dd73 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:23.015950 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c5af9f-bb16-4668-aa39-83463346b1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:23.016619 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c5af9f-bb16-4668-aa39-83463346b1f7 None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:23.016894 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c5af9f-bb16-4668-aa39-83463346b1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:23.017162 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c5af9f-bb16-4668-aa39-83463346b1f7 None admin] PATH_INFO: `/v3/projects/4251d157536c4934bf6f66268f3037a9/groups/0109f2c831fd421582207689515d2d91/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:23.019088 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c5af9f-bb16-4668-aa39-83463346b1f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=4251d157536c4934bf6f66268f3037a9, group_id=0109f2c831fd421582207689515d2d91, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:23.020545 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c5af9f-bb16-4668-aa39-83463346b1f7 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:23.038959 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 191/387] 10.222.0.52 () {58 vars in 1442 bytes} [Wed Sep 28 15:28:22 2022] PUT /identity/v3/projects/4251d157536c4934bf6f66268f3037a9/groups/0109f2c831fd421582207689515d2d91/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:28:23.223478 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed7529-25fa-4dfc-af0e-ce7d883b2b2c None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:23.223860 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed7529-25fa-4dfc-af0e-ce7d883b2b2c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:23.224035 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed7529-25fa-4dfc-af0e-ce7d883b2b2c None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:23.225774 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 197/388] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:23.235975 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af72c47-b07a-4a5d-ba83-142e4234d489 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:23.236273 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af72c47-b07a-4a5d-ba83-142e4234d489 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:23.236525 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af72c47-b07a-4a5d-ba83-142e4234d489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:23.263001 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-0af72c47-b07a-4a5d-ba83-142e4234d489 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:23.264237 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-0af72c47-b07a-4a5d-ba83-142e4234d489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:23.362537 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 192/389] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:23.779274 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b002705-9c18-418c-ba6a-fb75d4393ba4 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:23.779790 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b002705-9c18-418c-ba6a-fb75d4393ba4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:23.779790 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b002705-9c18-418c-ba6a-fb75d4393ba4 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:23.781281 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 198/390] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:23.786402 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e6976a-b147-499e-8551-a9f300c8e8ac None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:23.786804 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e6976a-b147-499e-8551-a9f300c8e8ac None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:23.786974 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e6976a-b147-499e-8551-a9f300c8e8ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:23.814807 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-17e6976a-b147-499e-8551-a9f300c8e8ac None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:23.815986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-17e6976a-b147-499e-8551-a9f300c8e8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:23.835474 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:23.846959 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b002705-9c18-418c-ba6a-fb75d4393ba4 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:23.906205 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 193/391] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:24.033517 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b002705-9c18-418c-ba6a-fb75d4393ba4 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:24.080654 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce65b44-79a6-44a4-9ccb-46c9fbb5f70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:24.081286 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce65b44-79a6-44a4-9ccb-46c9fbb5f70e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:24.081562 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce65b44-79a6-44a4-9ccb-46c9fbb5f70e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:24.081832 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce65b44-79a6-44a4-9ccb-46c9fbb5f70e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:24.095735 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce65b44-79a6-44a4-9ccb-46c9fbb5f70e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:24.100679 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-cce65b44-79a6-44a4-9ccb-46c9fbb5f70e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:28:24.100679 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:24.100679 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:24.100679 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:24.100679 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:24.100679 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:24.100679 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:24.100679 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:24.100679 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:24.100679 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:24.100679 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:24.100679 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:24.100679 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:24.100679 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:24.100679 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:24.100679 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:24.101585 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:24.101585 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:24.101585 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:24.101585 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:24.101585 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:24.101585 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:24.101585 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:24.101585 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:24.101585 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:24.101585 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:24.101585 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:24.101585 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:24.101585 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:24.101585 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:24.101585 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:24.101585 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:24.102408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:24.102408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:24.102408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:24.102408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:24.102408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:24.102408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:24.102408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:24.102408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:24.102408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:24.102408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:24.102408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:28:24.102408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:24.102996 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 199/392] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 15:28:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:24.108052 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e6976a-b147-499e-8551-a9f300c8e8ac None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:24.202891 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a8f2f-53b0-48cd-93ff-79d48d59a1d0 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:24.203176 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a8f2f-53b0-48cd-93ff-79d48d59a1d0 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:24.203483 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a8f2f-53b0-48cd-93ff-79d48d59a1d0 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:24.205089 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 200/393] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:28:24.209960 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7957e1-6e32-4b8b-92ca-6658d733e7fd None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:24.210251 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7957e1-6e32-4b8b-92ca-6658d733e7fd None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:24.210532 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7957e1-6e32-4b8b-92ca-6658d733e7fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:24.240767 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e6976a-b147-499e-8551-a9f300c8e8ac None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:24.242945 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-6b7957e1-6e32-4b8b-92ca-6658d733e7fd None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:24.244784 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6b7957e1-6e32-4b8b-92ca-6658d733e7fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:24.287446 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:24.299288 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c53c0c-0bfc-451c-af6c-7aee4ac5efe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:24.299989 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c53c0c-0bfc-451c-af6c-7aee4ac5efe1 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:24.300315 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c53c0c-0bfc-451c-af6c-7aee4ac5efe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:24.300588 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c53c0c-0bfc-451c-af6c-7aee4ac5efe1 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:24.301522 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c53c0c-0bfc-451c-af6c-7aee4ac5efe1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:24.301865 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c53c0c-0bfc-451c-af6c-7aee4ac5efe1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:24.305339 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c53c0c-0bfc-451c-af6c-7aee4ac5efe1 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:24.315790 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 194/394] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:28:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:24.316883 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c53c0c-0bfc-451c-af6c-7aee4ac5efe1 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:24.333960 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:24.352164 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 201/395] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:28:24.353085 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7957e1-6e32-4b8b-92ca-6658d733e7fd None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:24.436343 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c53c0c-0bfc-451c-af6c-7aee4ac5efe1 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:24.477474 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23b6141-9ab4-463e-b191-393252a5bcdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:24.478143 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b6141-9ab4-463e-b191-393252a5bcdc None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:24.478944 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b6141-9ab4-463e-b191-393252a5bcdc None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:24.479241 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b6141-9ab4-463e-b191-393252a5bcdc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:24.483937 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7957e1-6e32-4b8b-92ca-6658d733e7fd None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:24.491188 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23b6141-9ab4-463e-b191-393252a5bcdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:24.497541 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-b23b6141-9ab4-463e-b191-393252a5bcdc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:28:24.497541 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:24.497541 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:24.497541 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:24.497541 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:24.497541 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:24.497541 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:24.497541 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:24.497541 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:24.497541 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:24.497541 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:24.497541 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:24.497541 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:24.497541 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:24.497541 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:24.497541 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:24.498683 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:24.498683 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:24.498683 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:24.498683 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:24.498683 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:24.498683 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:24.498683 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:24.498683 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:24.498683 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:24.498683 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:24.498683 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:24.498683 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:24.498683 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:24.498683 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:24.498683 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:24.498683 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:24.499578 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:24.499578 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:24.499578 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:24.499578 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:24.499578 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:24.499578 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:24.499578 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:24.499578 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:24.499578 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:24.499578 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:24.499578 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:28:24.499578 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:24.499578 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 195/396] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:28:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:24.505989 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23b6141-9ab4-463e-b191-393252a5bcdc None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:24.528782 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c238321-033b-421e-bcce-75cccdb01acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:24.529694 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c238321-033b-421e-bcce-75cccdb01acf None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:24.530173 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c238321-033b-421e-bcce-75cccdb01acf None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:24.530593 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c238321-033b-421e-bcce-75cccdb01acf None admin] PATH_INFO: `/v3/groups/0109f2c831fd421582207689515d2d91` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:24.531895 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c238321-033b-421e-bcce-75cccdb01acf None admin] RBAC: Authorizing `identity:get_group(group_id=0109f2c831fd421582207689515d2d91)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:24.534333 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c238321-033b-421e-bcce-75cccdb01acf None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:24.537210 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 202/397] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Sep 28 15:28:24 2022] GET /identity/v3/groups/0109f2c831fd421582207689515d2d91 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:24.543477 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c238321-033b-421e-bcce-75cccdb01acf None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:24.589895 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:24.594905 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:24.630683 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23b6141-9ab4-463e-b191-393252a5bcdc None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:24.676190 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a009993-ac25-4993-ba8f-0b695f167558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:24.676919 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a009993-ac25-4993-ba8f-0b695f167558 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:24.677059 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a009993-ac25-4993-ba8f-0b695f167558 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:24.677321 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a009993-ac25-4993-ba8f-0b695f167558 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:24.678036 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a009993-ac25-4993-ba8f-0b695f167558 None admin] RBAC: Adding query filter params (name=member) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:24.678340 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a009993-ac25-4993-ba8f-0b695f167558 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:24.679776 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a009993-ac25-4993-ba8f-0b695f167558 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:24.693239 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 196/398] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 15:28:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:24.694516 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a009993-ac25-4993-ba8f-0b695f167558 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:24.706520 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c238321-033b-421e-bcce-75cccdb01acf None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:24.752254 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec57d045-4e87-4e3a-8b51-0a91382968a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:24.753671 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57d045-4e87-4e3a-8b51-0a91382968a1 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:24.754153 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57d045-4e87-4e3a-8b51-0a91382968a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:24.754732 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57d045-4e87-4e3a-8b51-0a91382968a1 None admin] PATH_INFO: `/v3/projects/4251d157536c4934bf6f66268f3037a9` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:24.756140 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec57d045-4e87-4e3a-8b51-0a91382968a1 None admin] RBAC: Authorizing `identity:get_project(project_id=4251d157536c4934bf6f66268f3037a9)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:24.758939 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec57d045-4e87-4e3a-8b51-0a91382968a1 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:24.767546 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 203/399] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:28:24 2022] GET /identity/v3/projects/4251d157536c4934bf6f66268f3037a9 => generated 298 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:24.772619 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec57d045-4e87-4e3a-8b51-0a91382968a1 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:24.843491 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a009993-ac25-4993-ba8f-0b695f167558 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:24.882430 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5a738a-2f8a-4b3e-8faa-b6fdfcb5c8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:24.883062 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a738a-2f8a-4b3e-8faa-b6fdfcb5c8fa None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:24.883339 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a738a-2f8a-4b3e-8faa-b6fdfcb5c8fa None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:24.883610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a738a-2f8a-4b3e-8faa-b6fdfcb5c8fa None admin] PATH_INFO: `/v3/groups/0109f2c831fd421582207689515d2d91` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:24.886378 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5a738a-2f8a-4b3e-8faa-b6fdfcb5c8fa None admin] RBAC: Authorizing `identity:get_group(group_id=0109f2c831fd421582207689515d2d91)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:24.887808 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5a738a-2f8a-4b3e-8faa-b6fdfcb5c8fa None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:24.889503 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 197/400] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Sep 28 15:28:24 2022] GET /identity/v3/groups/0109f2c831fd421582207689515d2d91 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:24.890671 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44308cf9-b1f6-4685-b99e-58ddc4daa205 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:24.890793 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44308cf9-b1f6-4685-b99e-58ddc4daa205 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:24.891015 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44308cf9-b1f6-4685-b99e-58ddc4daa205 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:24.892298 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 198/401] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:24.898026 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44308cf9-b1f6-4685-b99e-58ddc4daa205 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:24.900697 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:24.901542 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:24.913519 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec57d045-4e87-4e3a-8b51-0a91382968a1 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:24.968087 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ca9d97-d3f3-48fe-b155-9f9fdf8f603d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:24.968753 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ca9d97-d3f3-48fe-b155-9f9fdf8f603d None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:24.969028 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ca9d97-d3f3-48fe-b155-9f9fdf8f603d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:24.969298 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ca9d97-d3f3-48fe-b155-9f9fdf8f603d None admin] PATH_INFO: `/v3/projects/4251d157536c4934bf6f66268f3037a9/groups/0109f2c831fd421582207689515d2d91/roles/5038c8140b7145ceb7494210a5c9f9f0` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:24.971480 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ca9d97-d3f3-48fe-b155-9f9fdf8f603d None admin] RBAC: Authorizing `identity:create_grant(project_id=4251d157536c4934bf6f66268f3037a9, group_id=0109f2c831fd421582207689515d2d91, role_id=5038c8140b7145ceb7494210a5c9f9f0)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:24.972921 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ca9d97-d3f3-48fe-b155-9f9fdf8f603d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:24.990742 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 204/402] 10.222.0.52 () {58 vars in 1442 bytes} [Wed Sep 28 15:28:24 2022] PUT /identity/v3/projects/4251d157536c4934bf6f66268f3037a9/groups/0109f2c831fd421582207689515d2d91/roles/5038c8140b7145ceb7494210a5c9f9f0 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:28:24.991823 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc757b90-5048-4ac0-ae3f-b5929a8d5a20 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:24.992111 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc757b90-5048-4ac0-ae3f-b5929a8d5a20 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:24.992387 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc757b90-5048-4ac0-ae3f-b5929a8d5a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:25.019758 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-fc757b90-5048-4ac0-ae3f-b5929a8d5a20 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:25.021377 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-fc757b90-5048-4ac0-ae3f-b5929a8d5a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:25.033913 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44308cf9-b1f6-4685-b99e-58ddc4daa205 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:25.078138 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf0e6c7-2a27-43aa-a4cd-2897f8d05dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:25.078975 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0e6c7-2a27-43aa-a4cd-2897f8d05dec None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:25.079321 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0e6c7-2a27-43aa-a4cd-2897f8d05dec None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:25.079658 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0e6c7-2a27-43aa-a4cd-2897f8d05dec None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:25.093461 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf0e6c7-2a27-43aa-a4cd-2897f8d05dec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:25.099473 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-9cf0e6c7-2a27-43aa-a4cd-2897f8d05dec None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:28:25.099473 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:25.099473 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:25.099473 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:25.099473 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:25.099473 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:25.099473 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:25.099473 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:25.099473 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:25.099473 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:25.099473 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:25.099473 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:25.099473 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:25.099473 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:25.099473 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:25.099473 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:25.100391 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:25.100391 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:25.100391 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:25.100391 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:25.100391 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:25.100391 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:25.100391 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:25.100391 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:25.100391 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:25.100391 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:25.100391 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:25.100391 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:25.100391 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:25.100391 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:25.100391 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:25.100391 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:25.101401 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:25.101401 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:25.101401 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:25.101401 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:25.101401 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:25.101401 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:25.101401 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:25.101401 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:25.101401 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:25.101401 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:25.101401 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:28:25.101401 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:25.101401 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 199/403] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 15:28:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:25.102358 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf0e6c7-2a27-43aa-a4cd-2897f8d05dec None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:25.123206 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 205/404] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:25.124254 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc757b90-5048-4ac0-ae3f-b5929a8d5a20 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:25.223463 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:25.224221 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:25.244374 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc757b90-5048-4ac0-ae3f-b5929a8d5a20 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:25.284630 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf0e6c7-2a27-43aa-a4cd-2897f8d05dec None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:25.294904 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4afc5aa-1b50-4603-a939-f18228096da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:25.295590 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4afc5aa-1b50-4603-a939-f18228096da3 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:25.295931 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4afc5aa-1b50-4603-a939-f18228096da3 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:25.296264 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4afc5aa-1b50-4603-a939-f18228096da3 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:25.296994 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4afc5aa-1b50-4603-a939-f18228096da3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:25.297359 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4afc5aa-1b50-4603-a939-f18228096da3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:25.298819 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4afc5aa-1b50-4603-a939-f18228096da3 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:25.307519 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 206/405] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:28:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:25.313793 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4afc5aa-1b50-4603-a939-f18228096da3 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:25.355532 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a63dde-5d83-4da0-9044-ad29ee383bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:25.356076 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a63dde-5d83-4da0-9044-ad29ee383bea None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:25.356347 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a63dde-5d83-4da0-9044-ad29ee383bea None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:25.356606 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a63dde-5d83-4da0-9044-ad29ee383bea None admin] PATH_INFO: `/v3/projects/fbb3348b420b4f6ba5d171c387e5a403` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:25.358717 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a63dde-5d83-4da0-9044-ad29ee383bea None admin] RBAC: Authorizing `identity:get_project(project_id=fbb3348b420b4f6ba5d171c387e5a403)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:25.360149 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a63dde-5d83-4da0-9044-ad29ee383bea None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:25.361739 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 200/406] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:28:25 2022] GET /identity/v3/projects/fbb3348b420b4f6ba5d171c387e5a403 => generated 302 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:25.367502 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a63dde-5d83-4da0-9044-ad29ee383bea None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:25.487287 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a63dde-5d83-4da0-9044-ad29ee383bea None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:25.489484 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4afc5aa-1b50-4603-a939-f18228096da3 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:25.525589 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7a37a9-7f34-482a-ba8f-ea40d9bec880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:25.526618 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a37a9-7f34-482a-ba8f-ea40d9bec880 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:25.527102 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a37a9-7f34-482a-ba8f-ea40d9bec880 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:25.527652 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a37a9-7f34-482a-ba8f-ea40d9bec880 None admin] PATH_INFO: `/v3/groups/0109f2c831fd421582207689515d2d91` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:25.529020 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7a37a9-7f34-482a-ba8f-ea40d9bec880 None admin] RBAC: Authorizing `identity:get_group(group_id=0109f2c831fd421582207689515d2d91)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:25.530436 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92419833-a687-4555-82c3-ebb033418b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:25.530927 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7a37a9-7f34-482a-ba8f-ea40d9bec880 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:25.531161 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92419833-a687-4555-82c3-ebb033418b9c None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:25.531439 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92419833-a687-4555-82c3-ebb033418b9c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:25.531711 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92419833-a687-4555-82c3-ebb033418b9c None admin] PATH_INFO: `/v3/projects/fbb3348b420b4f6ba5d171c387e5a403/groups/0109f2c831fd421582207689515d2d91/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:25.533652 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92419833-a687-4555-82c3-ebb033418b9c None admin] RBAC: Authorizing `identity:create_grant(project_id=fbb3348b420b4f6ba5d171c387e5a403, group_id=0109f2c831fd421582207689515d2d91, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:25.533885 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 207/407] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Sep 28 15:28:25 2022] GET /identity/v3/groups/0109f2c831fd421582207689515d2d91 => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:25.534825 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7a37a9-7f34-482a-ba8f-ea40d9bec880 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:25.539242 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92419833-a687-4555-82c3-ebb033418b9c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:25.542168 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:25.556931 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:25.558917 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 201/408] 10.222.0.52 () {58 vars in 1442 bytes} [Wed Sep 28 15:28:25 2022] PUT /identity/v3/projects/fbb3348b420b4f6ba5d171c387e5a403/groups/0109f2c831fd421582207689515d2d91/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:28:25.560137 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92419833-a687-4555-82c3-ebb033418b9c None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:25.682144 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92419833-a687-4555-82c3-ebb033418b9c None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:25.725158 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff17e83-f4c9-44f0-ba7c-09f4cb4f437c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:25.726226 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff17e83-f4c9-44f0-ba7c-09f4cb4f437c None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:25.726705 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff17e83-f4c9-44f0-ba7c-09f4cb4f437c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:25.727174 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff17e83-f4c9-44f0-ba7c-09f4cb4f437c None admin] PATH_INFO: `/v3/projects/fbb3348b420b4f6ba5d171c387e5a403` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:25.730159 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff17e83-f4c9-44f0-ba7c-09f4cb4f437c None admin] RBAC: Authorizing `identity:get_project(project_id=fbb3348b420b4f6ba5d171c387e5a403)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:25.732673 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff17e83-f4c9-44f0-ba7c-09f4cb4f437c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:25.735488 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 202/409] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:28:25 2022] GET /identity/v3/projects/fbb3348b420b4f6ba5d171c387e5a403 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:25.741415 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff17e83-f4c9-44f0-ba7c-09f4cb4f437c None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:25.745638 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7a37a9-7f34-482a-ba8f-ea40d9bec880 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:25.816839 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef8ee07-8a65-483a-a3fb-ff33e3ce772a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:25.817483 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8ee07-8a65-483a-a3fb-ff33e3ce772a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:25.817763 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8ee07-8a65-483a-a3fb-ff33e3ce772a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:25.820204 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8ee07-8a65-483a-a3fb-ff33e3ce772a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:25.831834 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef8ee07-8a65-483a-a3fb-ff33e3ce772a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:25.837287 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-3ef8ee07-8a65-483a-a3fb-ff33e3ce772a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:28:25.837287 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:25.837287 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:25.837287 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:25.837287 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:25.837287 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:25.837287 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:25.837287 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:25.837287 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:25.837287 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:25.837287 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:25.837287 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:25.837287 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:25.837287 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:25.837287 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:25.837287 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:25.838264 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:25.838264 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:25.838264 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:25.838264 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:25.838264 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:25.838264 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:25.838264 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:25.838264 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:25.838264 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:25.838264 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:25.838264 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:25.838264 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:25.838264 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:25.838264 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:25.838264 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:25.838264 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:25.839085 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:25.839085 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:25.839085 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:25.839085 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:25.839085 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:25.839085 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:25.839085 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:25.839085 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:25.839085 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:25.839085 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:25.839085 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:28:25.839085 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:25.839715 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 208/410] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 15:28:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:25.844869 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef8ee07-8a65-483a-a3fb-ff33e3ce772a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:25.845481 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:25.894421 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff17e83-f4c9-44f0-ba7c-09f4cb4f437c None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:25.926680 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:25.934886 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278e11ed-5d31-4740-9274-7e1066f1ba6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:25.935865 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278e11ed-5d31-4740-9274-7e1066f1ba6a None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:25.936346 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278e11ed-5d31-4740-9274-7e1066f1ba6a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:25.936817 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278e11ed-5d31-4740-9274-7e1066f1ba6a None admin] PATH_INFO: `/v3/projects/fbb3348b420b4f6ba5d171c387e5a403/groups/0109f2c831fd421582207689515d2d91/roles/5038c8140b7145ceb7494210a5c9f9f0` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:25.939321 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278e11ed-5d31-4740-9274-7e1066f1ba6a None admin] RBAC: Authorizing `identity:create_grant(project_id=fbb3348b420b4f6ba5d171c387e5a403, group_id=0109f2c831fd421582207689515d2d91, role_id=5038c8140b7145ceb7494210a5c9f9f0)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:25.940962 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278e11ed-5d31-4740-9274-7e1066f1ba6a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:25.956595 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef8ee07-8a65-483a-a3fb-ff33e3ce772a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:25.967785 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 203/411] 10.222.0.52 () {58 vars in 1442 bytes} [Wed Sep 28 15:28:25 2022] PUT /identity/v3/projects/fbb3348b420b4f6ba5d171c387e5a403/groups/0109f2c831fd421582207689515d2d91/roles/5038c8140b7145ceb7494210a5c9f9f0 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:28:26.012463 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf6e8e9-29d1-4303-b49d-7c85552a7e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:26.013083 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf6e8e9-29d1-4303-b49d-7c85552a7e8d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:26.013363 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf6e8e9-29d1-4303-b49d-7c85552a7e8d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:26.013640 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf6e8e9-29d1-4303-b49d-7c85552a7e8d None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:26.014327 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf6e8e9-29d1-4303-b49d-7c85552a7e8d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:26.014629 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf6e8e9-29d1-4303-b49d-7c85552a7e8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:26.016011 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf6e8e9-29d1-4303-b49d-7c85552a7e8d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:26.031442 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 209/412] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:28:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:28:26.037143 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278e11ed-5d31-4740-9274-7e1066f1ba6a None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:26.186417 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278e11ed-5d31-4740-9274-7e1066f1ba6a None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:26.210181 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:26.229053 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e8668-d19c-43e8-ac4f-797ffaa35fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:26.230185 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e8668-d19c-43e8-ac4f-797ffaa35fe0 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:26.230708 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e8668-d19c-43e8-ac4f-797ffaa35fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:26.231189 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e8668-d19c-43e8-ac4f-797ffaa35fe0 None admin] PATH_INFO: `/v3/groups/ac94b5bc6e804e1db1bf63b9390ad35f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:26.232512 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e8668-d19c-43e8-ac4f-797ffaa35fe0 None admin] RBAC: Authorizing `identity:get_group(group_id=ac94b5bc6e804e1db1bf63b9390ad35f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:26.234269 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e8668-d19c-43e8-ac4f-797ffaa35fe0 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:26.236345 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 204/413] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Sep 28 15:28:26 2022] GET /identity/v3/groups/ac94b5bc6e804e1db1bf63b9390ad35f => generated 229 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:26.241503 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf6e8e9-29d1-4303-b49d-7c85552a7e8d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:26.327872 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e94dd6-383e-4c94-9a97-e9332520b94d None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:26.328293 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e94dd6-383e-4c94-9a97-e9332520b94d None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:26.328667 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e94dd6-383e-4c94-9a97-e9332520b94d None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:26.330369 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 205/414] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:28:26.339121 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e13ee-2ac6-4be9-85c6-9a822d6f80fe None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:26.339404 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e13ee-2ac6-4be9-85c6-9a822d6f80fe None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:26.339682 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e13ee-2ac6-4be9-85c6-9a822d6f80fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:26.359543 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf6e8e9-29d1-4303-b49d-7c85552a7e8d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:26.362044 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-753e13ee-2ac6-4be9-85c6-9a822d6f80fe None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:26.364123 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-753e13ee-2ac6-4be9-85c6-9a822d6f80fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:26.423539 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5936b85c-8e54-452c-a254-72506c6da6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:26.424176 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5936b85c-8e54-452c-a254-72506c6da6be None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:26.424456 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5936b85c-8e54-452c-a254-72506c6da6be None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:26.424731 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5936b85c-8e54-452c-a254-72506c6da6be None admin] PATH_INFO: `/v3/projects/8de950cb25d34fe8afed2528d2e112e6` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:26.426888 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5936b85c-8e54-452c-a254-72506c6da6be None admin] RBAC: Authorizing `identity:get_project(project_id=8de950cb25d34fe8afed2528d2e112e6)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:26.428354 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5936b85c-8e54-452c-a254-72506c6da6be None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:26.430017 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 210/415] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:28:26 2022] GET /identity/v3/projects/8de950cb25d34fe8afed2528d2e112e6 => generated 344 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:28:26.436329 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5936b85c-8e54-452c-a254-72506c6da6be None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:26.443953 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:26.476934 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 206/416] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:28:26.597296 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5936b85c-8e54-452c-a254-72506c6da6be None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:26.638838 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ad382-91d2-4b31-9cc8-3efb4fdf9925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:26.639865 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ad382-91d2-4b31-9cc8-3efb4fdf9925 None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:26.640335 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ad382-91d2-4b31-9cc8-3efb4fdf9925 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:26.640812 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ad382-91d2-4b31-9cc8-3efb4fdf9925 None admin] PATH_INFO: `/v3/projects/8de950cb25d34fe8afed2528d2e112e6/groups/ac94b5bc6e804e1db1bf63b9390ad35f/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:26.647079 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9ad382-91d2-4b31-9cc8-3efb4fdf9925 None admin] RBAC: Authorizing `identity:create_grant(project_id=8de950cb25d34fe8afed2528d2e112e6, group_id=ac94b5bc6e804e1db1bf63b9390ad35f, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:26.648751 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9ad382-91d2-4b31-9cc8-3efb4fdf9925 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:26.679176 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 211/417] 10.222.0.52 () {58 vars in 1442 bytes} [Wed Sep 28 15:28:26 2022] PUT /identity/v3/projects/8de950cb25d34fe8afed2528d2e112e6/groups/ac94b5bc6e804e1db1bf63b9390ad35f/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:28:26.826881 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753e13ee-2ac6-4be9-85c6-9a822d6f80fe None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:26.945531 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753e13ee-2ac6-4be9-85c6-9a822d6f80fe None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:26.966455 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:26.986680 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f475803-f00a-44e9-84ae-e7bdd792de25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:26.987371 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f475803-f00a-44e9-84ae-e7bdd792de25 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:26.987654 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f475803-f00a-44e9-84ae-e7bdd792de25 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:26.987924 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f475803-f00a-44e9-84ae-e7bdd792de25 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:26.999048 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f475803-f00a-44e9-84ae-e7bdd792de25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:27.003998 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-9f475803-f00a-44e9-84ae-e7bdd792de25 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:28:27.003998 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:27.003998 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:27.003998 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:27.003998 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:27.003998 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:27.003998 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:27.003998 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:27.003998 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:27.003998 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:27.003998 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:27.003998 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:27.003998 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:27.003998 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:27.003998 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:27.003998 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:27.004885 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:27.004885 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:27.004885 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:27.004885 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:27.004885 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:27.004885 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:27.004885 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:27.004885 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:27.004885 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:27.004885 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:27.004885 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:27.004885 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:27.004885 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:27.004885 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:27.004885 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:27.004885 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:27.005674 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:27.005674 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:27.005674 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:27.005674 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:27.005674 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:27.005674 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:27.005674 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:27.005674 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:27.005674 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:27.005674 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:27.005674 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:28:27.005674 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:27.005674 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 207/418] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:28:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:27.011570 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ad382-91d2-4b31-9cc8-3efb4fdf9925 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:27.081048 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:27.122308 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ad382-91d2-4b31-9cc8-3efb4fdf9925 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:27.158044 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa6fed0-b55d-45a3-9af6-105bcca37a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:27.158660 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa6fed0-b55d-45a3-9af6-105bcca37a6e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:27.158943 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa6fed0-b55d-45a3-9af6-105bcca37a6e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:27.159213 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa6fed0-b55d-45a3-9af6-105bcca37a6e None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:27.159874 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa6fed0-b55d-45a3-9af6-105bcca37a6e None admin] RBAC: Adding query filter params (name=member) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:27.160173 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa6fed0-b55d-45a3-9af6-105bcca37a6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:27.161603 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa6fed0-b55d-45a3-9af6-105bcca37a6e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:27.170873 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 212/419] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 15:28:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:27.176601 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f475803-f00a-44e9-84ae-e7bdd792de25 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:27.305649 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f475803-f00a-44e9-84ae-e7bdd792de25 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:27.346228 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86de72d-b036-41e1-b56c-0361c88ee34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:27.346766 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86de72d-b036-41e1-b56c-0361c88ee34a None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:27.347147 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86de72d-b036-41e1-b56c-0361c88ee34a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:27.347425 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86de72d-b036-41e1-b56c-0361c88ee34a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:27.412355 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:27.449231 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86de72d-b036-41e1-b56c-0361c88ee34a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:27.450696 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86de72d-b036-41e1-b56c-0361c88ee34a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:27.553866 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 208/420] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:28:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:28:27.559861 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa6fed0-b55d-45a3-9af6-105bcca37a6e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:27.703704 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa6fed0-b55d-45a3-9af6-105bcca37a6e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:27.734390 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:27.738860 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64c85aa-a41d-46c7-b5e3-8df9a4d23185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:27.739916 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64c85aa-a41d-46c7-b5e3-8df9a4d23185 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:27.740200 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64c85aa-a41d-46c7-b5e3-8df9a4d23185 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:27.740478 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64c85aa-a41d-46c7-b5e3-8df9a4d23185 None admin] PATH_INFO: `/v3/projects/4251d157536c4934bf6f66268f3037a9` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:27.741814 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64c85aa-a41d-46c7-b5e3-8df9a4d23185 None admin] RBAC: Authorizing `identity:get_project(project_id=4251d157536c4934bf6f66268f3037a9)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:27.743312 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64c85aa-a41d-46c7-b5e3-8df9a4d23185 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:27.750351 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 213/421] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:28:27 2022] GET /identity/v3/projects/4251d157536c4934bf6f66268f3037a9 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:27.751797 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86de72d-b036-41e1-b56c-0361c88ee34a None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:27.869840 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:27.872968 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b1a63-5593-420f-a556-25e539683be1 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:27.874126 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b1a63-5593-420f-a556-25e539683be1 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:27.874126 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b1a63-5593-420f-a556-25e539683be1 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:27.875130 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 214/422] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:27.880195 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c146a-a03e-4d13-afea-9df4a006ec97 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:27.880480 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c146a-a03e-4d13-afea-9df4a006ec97 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:27.880758 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c146a-a03e-4d13-afea-9df4a006ec97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:27.894593 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86de72d-b036-41e1-b56c-0361c88ee34a None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:27.912091 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-7f3c146a-a03e-4d13-afea-9df4a006ec97 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:27.913279 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7f3c146a-a03e-4d13-afea-9df4a006ec97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:27.932899 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6067b0da-59b6-4873-a276-9ae73cf07ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:27.933525 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6067b0da-59b6-4873-a276-9ae73cf07ab7 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:27.933797 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6067b0da-59b6-4873-a276-9ae73cf07ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:27.934091 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6067b0da-59b6-4873-a276-9ae73cf07ab7 None admin] PATH_INFO: `/v3/groups/0109f2c831fd421582207689515d2d91` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:27.935199 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6067b0da-59b6-4873-a276-9ae73cf07ab7 None admin] RBAC: Authorizing `identity:get_group(group_id=0109f2c831fd421582207689515d2d91)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:27.936613 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6067b0da-59b6-4873-a276-9ae73cf07ab7 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:27.938485 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 209/423] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Sep 28 15:28:27 2022] GET /identity/v3/groups/0109f2c831fd421582207689515d2d91 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:27.946783 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6067b0da-59b6-4873-a276-9ae73cf07ab7 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:28.083502 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6067b0da-59b6-4873-a276-9ae73cf07ab7 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:28.084755 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 215/424] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:28:28.130483 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff30fe5-6a46-43ce-a670-b39110805225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:28.131765 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff30fe5-6a46-43ce-a670-b39110805225 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:28.132253 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff30fe5-6a46-43ce-a670-b39110805225 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:28.132780 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff30fe5-6a46-43ce-a670-b39110805225 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:28.133709 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff30fe5-6a46-43ce-a670-b39110805225 None admin] RBAC: Adding query filter params (group.id=0109f2c831fd421582207689515d2d91, role.id=e152a36b5a1b470b912b1d705462f512, scope.project.id=4251d157536c4934bf6f66268f3037a9) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:28.134266 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff30fe5-6a46-43ce-a670-b39110805225 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:28.136565 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff30fe5-6a46-43ce-a670-b39110805225 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:28.146618 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 210/425] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Sep 28 15:28:27 2022] GET /identity/v3/role_assignments?group.id=0109f2c831fd421582207689515d2d91&scope.project.id=4251d157536c4934bf6f66268f3037a9&role.id=e152a36b5a1b470b912b1d705462f512 => generated 611 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:28.391224 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3c146a-a03e-4d13-afea-9df4a006ec97 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:28.435218 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edadeb02-3e01-462a-a7c5-9c283a5ed2cf None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:28.435559 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edadeb02-3e01-462a-a7c5-9c283a5ed2cf None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:28.435756 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edadeb02-3e01-462a-a7c5-9c283a5ed2cf None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:28.437345 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 211/426] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:28:28.442338 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549a527-e6ec-4271-9073-f7e2d81cb610 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:28.442614 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549a527-e6ec-4271-9073-f7e2d81cb610 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:28.442883 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549a527-e6ec-4271-9073-f7e2d81cb610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:28.463402 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-7549a527-e6ec-4271-9073-f7e2d81cb610 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:28.464690 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7549a527-e6ec-4271-9073-f7e2d81cb610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:28.466940 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:28.515288 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3c146a-a03e-4d13-afea-9df4a006ec97 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:28.552909 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:28.567995 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fb180c-36db-4f20-a53a-f3144795002a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:28.568640 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb180c-36db-4f20-a53a-f3144795002a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:28.568919 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb180c-36db-4f20-a53a-f3144795002a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:28.569186 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb180c-36db-4f20-a53a-f3144795002a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:28.571929 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 212/427] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:28:28.581210 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fb180c-36db-4f20-a53a-f3144795002a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:28.586134 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-b8fb180c-36db-4f20-a53a-f3144795002a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:28:28.586134 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:28.586134 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:28.586134 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:28.586134 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:28.586134 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:28.586134 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:28.586134 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:28.586134 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:28.586134 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:28.586134 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:28.586134 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:28.586134 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:28.586134 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:28.586134 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:28.586134 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:28.586993 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:28.586993 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:28.586993 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:28.586993 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:28.586993 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:28.586993 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:28.586993 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:28.586993 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:28.586993 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:28.586993 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:28.586993 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:28.586993 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:28.586993 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:28.586993 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:28.586993 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:28.586993 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:28.588320 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:28.588320 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:28.588320 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:28.588320 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:28.588320 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:28.588320 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:28.588320 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:28.588320 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:28.588320 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:28.588320 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:28.588320 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:28:28.588320 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:28.588320 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 216/428] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 15:28:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:28.593735 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549a527-e6ec-4271-9073-f7e2d81cb610 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:28.645014 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6fbf45-13a8-46f7-98e1-df866781fbc5 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:28.645274 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6fbf45-13a8-46f7-98e1-df866781fbc5 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:28.645493 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6fbf45-13a8-46f7-98e1-df866781fbc5 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:28.646976 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 217/429] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:28.653604 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7adbd0-7984-4781-9786-677a9f917e08 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:28.653906 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7adbd0-7984-4781-9786-677a9f917e08 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:28.654225 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7adbd0-7984-4781-9786-677a9f917e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:28.675937 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-8c7adbd0-7984-4781-9786-677a9f917e08 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:28.678041 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8c7adbd0-7984-4781-9786-677a9f917e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:28.715968 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549a527-e6ec-4271-9073-f7e2d81cb610 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:28.760460 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de74796c-81a4-458c-9a76-d581d895dbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:28.761436 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de74796c-81a4-458c-9a76-d581d895dbd8 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:28.761954 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de74796c-81a4-458c-9a76-d581d895dbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:28.762463 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de74796c-81a4-458c-9a76-d581d895dbd8 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:28.763498 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de74796c-81a4-458c-9a76-d581d895dbd8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:28.763992 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de74796c-81a4-458c-9a76-d581d895dbd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:28.765062 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 218/430] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:28:28.765864 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de74796c-81a4-458c-9a76-d581d895dbd8 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:28.775736 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:28.781476 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 213/431] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:28:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:28.787616 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7adbd0-7984-4781-9786-677a9f917e08 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:28.891795 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de74796c-81a4-458c-9a76-d581d895dbd8 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:28.904012 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7adbd0-7984-4781-9786-677a9f917e08 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:28.924694 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:28.957280 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79101af-7fd6-46b5-8b3d-656f8e15b626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:28.957919 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79101af-7fd6-46b5-8b3d-656f8e15b626 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:28.958200 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79101af-7fd6-46b5-8b3d-656f8e15b626 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:28.958470 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79101af-7fd6-46b5-8b3d-656f8e15b626 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:29.015250 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de74796c-81a4-458c-9a76-d581d895dbd8 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:29.046223 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79101af-7fd6-46b5-8b3d-656f8e15b626 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:29.048364 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79101af-7fd6-46b5-8b3d-656f8e15b626 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:29.058395 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d8077-b960-4d13-88e9-01ff59e8daff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:29.059036 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d8077-b960-4d13-88e9-01ff59e8daff None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:29.059312 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d8077-b960-4d13-88e9-01ff59e8daff None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:29.059583 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d8077-b960-4d13-88e9-01ff59e8daff None admin] PATH_INFO: `/v3/roles/member` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:29.070690 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274d8077-b960-4d13-88e9-01ff59e8daff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:29.076425 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-274d8077-b960-4d13-88e9-01ff59e8daff None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:28:29.076425 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:29.076425 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:29.076425 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:29.076425 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:29.076425 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:29.076425 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:29.076425 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:29.076425 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:29.076425 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:29.076425 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:29.076425 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:29.076425 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:29.076425 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:29.076425 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:29.076425 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:29.077316 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:29.077316 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:29.077316 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:29.077316 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:29.077316 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:29.077316 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:29.077316 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:29.077316 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:29.077316 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:29.077316 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:29.077316 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:29.077316 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:29.077316 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:29.077316 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:29.077316 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:29.077316 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:29.078155 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:29.078155 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:29.078155 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:29.078155 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:29.078155 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:29.078155 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:29.078155 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:29.078155 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:29.078155 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:29.078155 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:29.078155 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 28 15:28:29.078155 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:29.080745 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 214/432] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:28:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:29.081696 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d8077-b960-4d13-88e9-01ff59e8daff None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:29.147902 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:29.180161 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 219/433] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:28:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 28 15:28:29.182241 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79101af-7fd6-46b5-8b3d-656f8e15b626 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:29.195439 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d8077-b960-4d13-88e9-01ff59e8daff None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:29.198842 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:29.234301 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563b2990-ef7c-417f-87bb-51078fd671ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:29.235211 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b2990-ef7c-417f-87bb-51078fd671ba None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:29.235490 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b2990-ef7c-417f-87bb-51078fd671ba None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:29.235759 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b2990-ef7c-417f-87bb-51078fd671ba None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:29.247730 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563b2990-ef7c-417f-87bb-51078fd671ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:29.253070 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-563b2990-ef7c-417f-87bb-51078fd671ba None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:28:29.253070 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:29.253070 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:29.253070 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:29.253070 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:29.253070 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:29.253070 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:29.253070 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:29.253070 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:29.253070 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:29.253070 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:29.253070 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:29.253070 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:29.253070 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:29.253070 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:29.253070 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:29.253980 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:29.253980 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:29.253980 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:29.253980 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:29.253980 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:29.253980 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:29.253980 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:29.253980 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:29.253980 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:29.253980 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:29.253980 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:29.253980 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:29.253980 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:29.253980 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:29.253980 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:29.253980 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:29.254935 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:29.254935 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:29.254935 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:29.254935 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:29.254935 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:29.254935 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:29.254935 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:29.254935 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:29.254935 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:29.254935 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:29.254935 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:28:29.254935 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:29.259096 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 215/434] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 15:28:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:29.260167 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563b2990-ef7c-417f-87bb-51078fd671ba None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:29.304729 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79101af-7fd6-46b5-8b3d-656f8e15b626 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:29.339444 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866aec0b-d60e-40b0-b0d4-eefc4d1642fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:29.340050 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866aec0b-d60e-40b0-b0d4-eefc4d1642fe None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:29.340330 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866aec0b-d60e-40b0-b0d4-eefc4d1642fe None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:29.340602 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866aec0b-d60e-40b0-b0d4-eefc4d1642fe None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:29.341262 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866aec0b-d60e-40b0-b0d4-eefc4d1642fe None admin] RBAC: Adding query filter params (name=member) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:29.341564 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866aec0b-d60e-40b0-b0d4-eefc4d1642fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:29.342980 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866aec0b-d60e-40b0-b0d4-eefc4d1642fe None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:29.354144 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 220/435] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 15:28:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:29.354144 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866aec0b-d60e-40b0-b0d4-eefc4d1642fe None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:29.384081 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563b2990-ef7c-417f-87bb-51078fd671ba None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:29.405639 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:29.424063 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1496e36b-45f1-42e1-a1d5-b233b0a6305a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:29.424689 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496e36b-45f1-42e1-a1d5-b233b0a6305a None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:29.424964 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496e36b-45f1-42e1-a1d5-b233b0a6305a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:29.425230 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496e36b-45f1-42e1-a1d5-b233b0a6305a None admin] PATH_INFO: `/v3/projects/4251d157536c4934bf6f66268f3037a9` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:29.426786 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1496e36b-45f1-42e1-a1d5-b233b0a6305a None admin] RBAC: Authorizing `identity:get_project(project_id=4251d157536c4934bf6f66268f3037a9)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:29.428256 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1496e36b-45f1-42e1-a1d5-b233b0a6305a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:29.434827 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 216/436] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:28:29 2022] GET /identity/v3/projects/4251d157536c4934bf6f66268f3037a9 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:29.434827 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1496e36b-45f1-42e1-a1d5-b233b0a6305a None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:29.454379 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:29.478991 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866aec0b-d60e-40b0-b0d4-eefc4d1642fe None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:29.514608 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e3dd59-7450-4a33-9087-ee25a73cb9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:29.515201 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3dd59-7450-4a33-9087-ee25a73cb9b9 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:29.515479 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3dd59-7450-4a33-9087-ee25a73cb9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:29.515752 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3dd59-7450-4a33-9087-ee25a73cb9b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:29.516392 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e3dd59-7450-4a33-9087-ee25a73cb9b9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:29.516693 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e3dd59-7450-4a33-9087-ee25a73cb9b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:29.518080 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e3dd59-7450-4a33-9087-ee25a73cb9b9 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:29.525643 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 221/437] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:28:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:29.526432 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e3dd59-7450-4a33-9087-ee25a73cb9b9 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:29.537883 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1496e36b-45f1-42e1-a1d5-b233b0a6305a None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:29.569950 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb557f4-4262-43df-b1f6-68c3ad849c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:29.570540 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb557f4-4262-43df-b1f6-68c3ad849c60 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:29.570811 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb557f4-4262-43df-b1f6-68c3ad849c60 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:29.571073 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb557f4-4262-43df-b1f6-68c3ad849c60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:29.624675 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:29.625166 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e3dd59-7450-4a33-9087-ee25a73cb9b9 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:29.639762 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb557f4-4262-43df-b1f6-68c3ad849c60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:29.641509 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb557f4-4262-43df-b1f6-68c3ad849c60 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:29.662962 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e061e-f757-45bd-bae5-10fbe3629aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:29.663591 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e061e-f757-45bd-bae5-10fbe3629aa7 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:29.663866 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e061e-f757-45bd-bae5-10fbe3629aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:29.664133 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e061e-f757-45bd-bae5-10fbe3629aa7 None admin] PATH_INFO: `/v3/groups/0109f2c831fd421582207689515d2d91` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:29.665240 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78e061e-f757-45bd-bae5-10fbe3629aa7 None admin] RBAC: Authorizing `identity:get_group(group_id=0109f2c831fd421582207689515d2d91)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:29.667195 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78e061e-f757-45bd-bae5-10fbe3629aa7 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:29.668736 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 222/438] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Sep 28 15:28:29 2022] GET /identity/v3/groups/0109f2c831fd421582207689515d2d91 => generated 226 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:29.669822 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd19db-37d5-4c57-85b4-67cf325965e6 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:29.670128 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd19db-37d5-4c57-85b4-67cf325965e6 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:29.670401 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd19db-37d5-4c57-85b4-67cf325965e6 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:29.671727 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 223/439] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:29.672256 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbd19db-37d5-4c57-85b4-67cf325965e6 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:29.682478 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:29.751984 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 217/440] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:28:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:28:29.753003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb557f4-4262-43df-b1f6-68c3ad849c60 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:29.779286 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbd19db-37d5-4c57-85b4-67cf325965e6 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:29.811908 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbd5f45-cce1-4706-8ecc-57aa1a06f325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:29.812497 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbd5f45-cce1-4706-8ecc-57aa1a06f325 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:29.812776 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbd5f45-cce1-4706-8ecc-57aa1a06f325 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:29.813053 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbd5f45-cce1-4706-8ecc-57aa1a06f325 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:29.839879 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:29.852534 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb557f4-4262-43df-b1f6-68c3ad849c60 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:29.879530 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbd5f45-cce1-4706-8ecc-57aa1a06f325 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:29.881006 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbd5f45-cce1-4706-8ecc-57aa1a06f325 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:29.886098 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9577fb1-99a8-4fe0-a559-f9b754839f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:29.886701 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9577fb1-99a8-4fe0-a559-f9b754839f5b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:29.886975 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9577fb1-99a8-4fe0-a559-f9b754839f5b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:29.887245 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9577fb1-99a8-4fe0-a559-f9b754839f5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:29.887951 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9577fb1-99a8-4fe0-a559-f9b754839f5b None admin] RBAC: Adding query filter params (group.id=0109f2c831fd421582207689515d2d91, role.id=5038c8140b7145ceb7494210a5c9f9f0, scope.project.id=4251d157536c4934bf6f66268f3037a9) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:29.888252 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9577fb1-99a8-4fe0-a559-f9b754839f5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:29.889291 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:29.889656 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9577fb1-99a8-4fe0-a559-f9b754839f5b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:29.895350 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 218/441] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Sep 28 15:28:29 2022] GET /identity/v3/role_assignments?group.id=0109f2c831fd421582207689515d2d91&scope.project.id=4251d157536c4934bf6f66268f3037a9&role.id=5038c8140b7145ceb7494210a5c9f9f0 => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:29.896536 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476e7de-d6be-4012-a37e-61cf808b66bf None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:29.896820 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476e7de-d6be-4012-a37e-61cf808b66bf None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:29.897086 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476e7de-d6be-4012-a37e-61cf808b66bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:29.919295 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-c476e7de-d6be-4012-a37e-61cf808b66bf None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:29.920508 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c476e7de-d6be-4012-a37e-61cf808b66bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:30.005963 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 219/442] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:30.005963 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c476e7de-d6be-4012-a37e-61cf808b66bf None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:30.006487 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 224/443] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:28:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:28:30.006487 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbd5f45-cce1-4706-8ecc-57aa1a06f325 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:30.117512 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:30.140559 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbd5f45-cce1-4706-8ecc-57aa1a06f325 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:30.141641 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c476e7de-d6be-4012-a37e-61cf808b66bf None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:30.151046 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:30.188367 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9bf220-3973-4d01-a75d-7075e2f8a5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:30.189097 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9bf220-3973-4d01-a75d-7075e2f8a5cc None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:30.189533 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9bf220-3973-4d01-a75d-7075e2f8a5cc None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:30.189963 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f388e79e-dbc7-437a-9a97-b674707a45f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:30.190097 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9bf220-3973-4d01-a75d-7075e2f8a5cc None admin] PATH_INFO: `/v3/projects/fbb3348b420b4f6ba5d171c387e5a403` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:30.190836 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388e79e-dbc7-437a-9a97-b674707a45f9 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:30.191105 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9bf220-3973-4d01-a75d-7075e2f8a5cc None admin] RBAC: Authorizing `identity:get_project(project_id=fbb3348b420b4f6ba5d171c387e5a403)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:30.191415 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388e79e-dbc7-437a-9a97-b674707a45f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:30.191816 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388e79e-dbc7-437a-9a97-b674707a45f9 None admin] PATH_INFO: `/v3/projects/fbb3348b420b4f6ba5d171c387e5a403` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:30.192545 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9bf220-3973-4d01-a75d-7075e2f8a5cc None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:30.193204 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f388e79e-dbc7-437a-9a97-b674707a45f9 None admin] RBAC: Authorizing `identity:get_project(project_id=fbb3348b420b4f6ba5d171c387e5a403)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:30.194829 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f388e79e-dbc7-437a-9a97-b674707a45f9 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:30.197790 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 220/445] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:28:29 2022] GET /identity/v3/projects/fbb3348b420b4f6ba5d171c387e5a403 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:30.197930 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 225/445] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:28:29 2022] GET /identity/v3/projects/fbb3348b420b4f6ba5d171c387e5a403 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:30.200089 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9bf220-3973-4d01-a75d-7075e2f8a5cc None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:30.203006 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f388e79e-dbc7-437a-9a97-b674707a45f9 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:30.332135 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f388e79e-dbc7-437a-9a97-b674707a45f9 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:30.335178 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9bf220-3973-4d01-a75d-7075e2f8a5cc None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:30.366719 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e916440-b91c-44e4-821e-1fd99a8cd08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:30.367334 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e916440-b91c-44e4-821e-1fd99a8cd08e None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:30.367608 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e916440-b91c-44e4-821e-1fd99a8cd08e None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:30.367878 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e916440-b91c-44e4-821e-1fd99a8cd08e None admin] PATH_INFO: `/v3/groups/0109f2c831fd421582207689515d2d91` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:30.368991 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e916440-b91c-44e4-821e-1fd99a8cd08e None admin] RBAC: Authorizing `identity:get_group(group_id=0109f2c831fd421582207689515d2d91)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:30.369865 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dc9f52-aeca-4619-a436-81bcc5f35083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:30.370430 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e916440-b91c-44e4-821e-1fd99a8cd08e None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:30.370518 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc9f52-aeca-4619-a436-81bcc5f35083 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:30.370747 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc9f52-aeca-4619-a436-81bcc5f35083 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:30.371014 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc9f52-aeca-4619-a436-81bcc5f35083 None admin] PATH_INFO: `/v3/groups/0109f2c831fd421582207689515d2d91` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:30.371960 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 221/446] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Sep 28 15:28:30 2022] GET /identity/v3/groups/0109f2c831fd421582207689515d2d91 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:28:30.372083 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dc9f52-aeca-4619-a436-81bcc5f35083 None admin] RBAC: Authorizing `identity:get_group(group_id=0109f2c831fd421582207689515d2d91)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:30.372692 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e916440-b91c-44e4-821e-1fd99a8cd08e None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:30.373487 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dc9f52-aeca-4619-a436-81bcc5f35083 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:30.375054 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 226/447] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Sep 28 15:28:30 2022] GET /identity/v3/groups/0109f2c831fd421582207689515d2d91 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:30.377616 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dc9f52-aeca-4619-a436-81bcc5f35083 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:30.396368 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:30.420244 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:30.474159 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dc9f52-aeca-4619-a436-81bcc5f35083 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:30.474938 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e916440-b91c-44e4-821e-1fd99a8cd08e None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:30.507990 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef39a414-4751-4f15-b090-bca7989e53a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:30.508138 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabac2b5-ca14-4b43-bfc7-771d7bd01894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:30.508596 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39a414-4751-4f15-b090-bca7989e53a2 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:30.508693 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabac2b5-ca14-4b43-bfc7-771d7bd01894 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:30.508868 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39a414-4751-4f15-b090-bca7989e53a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:30.508958 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabac2b5-ca14-4b43-bfc7-771d7bd01894 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:30.509133 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39a414-4751-4f15-b090-bca7989e53a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:30.509225 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabac2b5-ca14-4b43-bfc7-771d7bd01894 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:30.509950 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabac2b5-ca14-4b43-bfc7-771d7bd01894 None admin] RBAC: Adding query filter params (group.id=0109f2c831fd421582207689515d2d91, role.id=e152a36b5a1b470b912b1d705462f512, scope.project.id=fbb3348b420b4f6ba5d171c387e5a403) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:30.510260 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabac2b5-ca14-4b43-bfc7-771d7bd01894 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:30.511696 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabac2b5-ca14-4b43-bfc7-771d7bd01894 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:30.517468 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 227/448] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Sep 28 15:28:30 2022] GET /identity/v3/role_assignments?group.id=0109f2c831fd421582207689515d2d91&scope.project.id=fbb3348b420b4f6ba5d171c387e5a403&role.id=e152a36b5a1b470b912b1d705462f512 => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:28:30.518339 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabac2b5-ca14-4b43-bfc7-771d7bd01894 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:30.519107 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef39a414-4751-4f15-b090-bca7989e53a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:30.524534 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-ef39a414-4751-4f15-b090-bca7989e53a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:28:30.524534 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:30.524534 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:30.524534 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:30.524534 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:30.524534 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:30.524534 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:30.524534 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:30.524534 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:30.524534 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:30.524534 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:30.524534 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:30.524534 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:30.524534 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:30.524534 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:30.524534 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:30.525681 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:30.525681 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:30.525681 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:30.525681 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:30.525681 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:30.525681 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:30.525681 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:30.525681 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:30.525681 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:30.525681 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:30.525681 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:30.525681 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:30.525681 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:30.525681 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:30.525681 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:30.525681 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:30.526752 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:30.526752 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:30.526752 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:30.526752 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:30.526752 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:30.526752 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:30.526752 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:30.526752 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:30.526752 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:30.526752 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:30.526752 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:28:30.526752 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:30.527845 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 222/449] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 15:28:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:30.535194 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef39a414-4751-4f15-b090-bca7989e53a2 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:30.631945 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:30.637654 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabac2b5-ca14-4b43-bfc7-771d7bd01894 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:30.645918 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:30.657993 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef39a414-4751-4f15-b090-bca7989e53a2 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:30.674576 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61199165-b243-4bb4-bad8-84ac33bacf0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:30.675180 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61199165-b243-4bb4-bad8-84ac33bacf0d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:30.675462 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61199165-b243-4bb4-bad8-84ac33bacf0d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:30.675731 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61199165-b243-4bb4-bad8-84ac33bacf0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:30.676464 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61199165-b243-4bb4-bad8-84ac33bacf0d None admin] RBAC: Adding query filter params (group.id=0109f2c831fd421582207689515d2d91, role.id=5038c8140b7145ceb7494210a5c9f9f0, scope.project.id=fbb3348b420b4f6ba5d171c387e5a403) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:30.676769 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61199165-b243-4bb4-bad8-84ac33bacf0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:30.678236 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61199165-b243-4bb4-bad8-84ac33bacf0d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:30.686394 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 228/450] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Sep 28 15:28:30 2022] GET /identity/v3/role_assignments?group.id=0109f2c831fd421582207689515d2d91&scope.project.id=fbb3348b420b4f6ba5d171c387e5a403&role.id=5038c8140b7145ceb7494210a5c9f9f0 => generated 611 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:30.699231 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a46fe-0cfc-468d-9aa6-a4e7fe55da41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:30.699942 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a46fe-0cfc-468d-9aa6-a4e7fe55da41 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:30.700216 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a46fe-0cfc-468d-9aa6-a4e7fe55da41 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:30.700484 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a46fe-0cfc-468d-9aa6-a4e7fe55da41 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:30.701155 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0a46fe-0cfc-468d-9aa6-a4e7fe55da41 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:30.701453 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0a46fe-0cfc-468d-9aa6-a4e7fe55da41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:30.702881 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0a46fe-0cfc-468d-9aa6-a4e7fe55da41 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:30.713104 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 223/451] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:28:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:28:30.718949 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61199165-b243-4bb4-bad8-84ac33bacf0d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:30.843050 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61199165-b243-4bb4-bad8-84ac33bacf0d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:30.882345 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9ae28d-f25f-417e-9edf-f6be6ece4507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:30.882879 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ae28d-f25f-417e-9edf-f6be6ece4507 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:30.883162 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ae28d-f25f-417e-9edf-f6be6ece4507 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:30.883433 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ae28d-f25f-417e-9edf-f6be6ece4507 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:30.928029 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:30.959243 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9ae28d-f25f-417e-9edf-f6be6ece4507 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:30.960896 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9ae28d-f25f-417e-9edf-f6be6ece4507 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:31.060342 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 229/452] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:28:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:28:31.066363 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a46fe-0cfc-468d-9aa6-a4e7fe55da41 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:31.164425 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a46fe-0cfc-468d-9aa6-a4e7fe55da41 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:31.196488 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e892276-c509-4fd5-872f-81e3c0f4e07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:31.197137 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e892276-c509-4fd5-872f-81e3c0f4e07b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:31.197410 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e892276-c509-4fd5-872f-81e3c0f4e07b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:31.197675 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e892276-c509-4fd5-872f-81e3c0f4e07b None admin] PATH_INFO: `/v3/projects/8de950cb25d34fe8afed2528d2e112e6` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:31.198787 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e892276-c509-4fd5-872f-81e3c0f4e07b None admin] RBAC: Authorizing `identity:get_project(project_id=8de950cb25d34fe8afed2528d2e112e6)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:31.200205 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e892276-c509-4fd5-872f-81e3c0f4e07b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:31.201778 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 224/453] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:28:31 2022] GET /identity/v3/projects/8de950cb25d34fe8afed2528d2e112e6 => generated 344 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:31.207313 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9ae28d-f25f-417e-9edf-f6be6ece4507 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:31.273191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:31.303986 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9ae28d-f25f-417e-9edf-f6be6ece4507 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:31.336337 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aee3562-7f2f-4ca3-b316-49991b4864a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:31.336957 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee3562-7f2f-4ca3-b316-49991b4864a6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:31.337235 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee3562-7f2f-4ca3-b316-49991b4864a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:31.337532 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aee3562-7f2f-4ca3-b316-49991b4864a6 None admin] PATH_INFO: `/v3/groups/ac94b5bc6e804e1db1bf63b9390ad35f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:31.338644 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aee3562-7f2f-4ca3-b316-49991b4864a6 None admin] RBAC: Authorizing `identity:get_group(group_id=ac94b5bc6e804e1db1bf63b9390ad35f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:31.340107 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aee3562-7f2f-4ca3-b316-49991b4864a6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:31.341666 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 230/454] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Sep 28 15:28:31 2022] GET /identity/v3/groups/ac94b5bc6e804e1db1bf63b9390ad35f => generated 229 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:31.347446 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e892276-c509-4fd5-872f-81e3c0f4e07b None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:31.358937 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:31.443463 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e892276-c509-4fd5-872f-81e3c0f4e07b None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:31.474254 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6f88c2-339e-4110-9ecd-09117bb9d925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:31.474832 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6f88c2-339e-4110-9ecd-09117bb9d925 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:31.475102 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6f88c2-339e-4110-9ecd-09117bb9d925 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:31.475364 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6f88c2-339e-4110-9ecd-09117bb9d925 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:31.476063 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6f88c2-339e-4110-9ecd-09117bb9d925 None admin] RBAC: Adding query filter params (group.id=ac94b5bc6e804e1db1bf63b9390ad35f, role.id=62d96ac9513c452894fc67b442cfc59f, scope.project.id=8de950cb25d34fe8afed2528d2e112e6) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:31.476364 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6f88c2-339e-4110-9ecd-09117bb9d925 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:31.477764 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6f88c2-339e-4110-9ecd-09117bb9d925 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:31.483680 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 225/455] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Sep 28 15:28:31 2022] GET /identity/v3/role_assignments?group.id=ac94b5bc6e804e1db1bf63b9390ad35f&scope.project.id=8de950cb25d34fe8afed2528d2e112e6&role.id=62d96ac9513c452894fc67b442cfc59f => generated 611 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:34.060491 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6964ef4-1871-483d-a726-347d55ab98af None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:34.061530 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6964ef4-1871-483d-a726-347d55ab98af None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:34.062037 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6964ef4-1871-483d-a726-347d55ab98af None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:34.063823 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 231/456] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:34.069113 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0634f7f-0579-4c27-8573-25b9d187a427 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:34.070003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0634f7f-0579-4c27-8573-25b9d187a427 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:34.070466 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0634f7f-0579-4c27-8573-25b9d187a427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:34.093733 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-a0634f7f-0579-4c27-8573-25b9d187a427 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:34.099270 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a0634f7f-0579-4c27-8573-25b9d187a427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:34.183130 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 226/457] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:34.503949 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c251b441-c25f-4ea3-910e-696e27e1d989 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:34.504620 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c251b441-c25f-4ea3-910e-696e27e1d989 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:34.505102 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c251b441-c25f-4ea3-910e-696e27e1d989 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:34.507196 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 232/458] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:34.512753 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180341f-258b-41bc-a5e6-c7b2bd4f2d60 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:34.513302 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180341f-258b-41bc-a5e6-c7b2bd4f2d60 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:34.513763 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180341f-258b-41bc-a5e6-c7b2bd4f2d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:34.524650 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:34.533732 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c251b441-c25f-4ea3-910e-696e27e1d989 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:34.543181 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-f180341f-258b-41bc-a5e6-c7b2bd4f2d60 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:34.544637 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-f180341f-258b-41bc-a5e6-c7b2bd4f2d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:34.644622 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 227/459] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:34.692576 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c251b441-c25f-4ea3-910e-696e27e1d989 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:34.731311 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ffc52a-fc8d-49ad-a3a4-f179cf5b7a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:34.732302 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ffc52a-fc8d-49ad-a3a4-f179cf5b7a45 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:34.732813 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ffc52a-fc8d-49ad-a3a4-f179cf5b7a45 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:34.733315 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ffc52a-fc8d-49ad-a3a4-f179cf5b7a45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:34.741544 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:34.745768 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ffc52a-fc8d-49ad-a3a4-f179cf5b7a45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:34.747564 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ffc52a-fc8d-49ad-a3a4-f179cf5b7a45 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:34.764668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-58ffc52a-fc8d-49ad-a3a4-f179cf5b7a45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:34.764668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:34.764668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:34.764668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:34.764668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:34.764668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:34.764668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:34.764668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:34.764668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:34.764668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:34.764668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:34.764668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:34.764668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:28:34.764668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:28:34.764668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:28:34.764668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:28:34.765574 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:34.765574 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:34.765574 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:34.765574 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:34.765574 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:34.765574 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:34.765574 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:34.765574 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:34.765574 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:34.765574 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:34.765574 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:34.765574 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:34.765574 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:34.765574 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:34.765574 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:34.765574 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:34.765574 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:28:34.766457 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:28:34.766457 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:34.766457 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:34.768522 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 233/460] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:34.774350 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f180341f-258b-41bc-a5e6-c7b2bd4f2d60 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:34.810718 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8bf369-1d6a-471d-bece-cc329ac351c9 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:34.811387 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8bf369-1d6a-471d-bece-cc329ac351c9 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:34.811885 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8bf369-1d6a-471d-bece-cc329ac351c9 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:34.813836 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 234/461] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:34.822552 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b9736-dd39-497c-b4fa-d54ff57dca45 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:34.823157 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b9736-dd39-497c-b4fa-d54ff57dca45 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:34.823655 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b9736-dd39-497c-b4fa-d54ff57dca45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:34.853883 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-a17b9736-dd39-497c-b4fa-d54ff57dca45 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:34.857187 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a17b9736-dd39-497c-b4fa-d54ff57dca45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:34.923619 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f180341f-258b-41bc-a5e6-c7b2bd4f2d60 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:34.969996 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:34.970939 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33461d1a-b23f-46bc-8a62-c311d03bb04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:34.971489 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33461d1a-b23f-46bc-8a62-c311d03bb04f None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:34.972242 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33461d1a-b23f-46bc-8a62-c311d03bb04f None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:34.972574 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33461d1a-b23f-46bc-8a62-c311d03bb04f None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:34.973615 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33461d1a-b23f-46bc-8a62-c311d03bb04f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:34.974072 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33461d1a-b23f-46bc-8a62-c311d03bb04f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:34.976341 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33461d1a-b23f-46bc-8a62-c311d03bb04f None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:34.981177 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 235/462] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:34.982766 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ffba3-4151-4b21-a2e9-eb542a9eab49 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:34.983197 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ffba3-4151-4b21-a2e9-eb542a9eab49 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:34.983633 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ffba3-4151-4b21-a2e9-eb542a9eab49 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:34.985374 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 236/463] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:34.989898 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041ffba3-4151-4b21-a2e9-eb542a9eab49 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:34.995809 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 228/464] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:28:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:34.997076 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5684b12-9d76-49bb-b004-29afd382b988 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:34.997356 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5684b12-9d76-49bb-b004-29afd382b988 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:34.997629 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5684b12-9d76-49bb-b004-29afd382b988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:35.022964 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-e5684b12-9d76-49bb-b004-29afd382b988 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:35.024079 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e5684b12-9d76-49bb-b004-29afd382b988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:35.107795 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 229/465] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:35.108619 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5684b12-9d76-49bb-b004-29afd382b988 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:35.165522 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041ffba3-4151-4b21-a2e9-eb542a9eab49 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:35.194161 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:35.210497 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea34cd29-1ca0-458b-9528-190151dd329f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:35.211343 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34cd29-1ca0-458b-9528-190151dd329f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:35.211760 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34cd29-1ca0-458b-9528-190151dd329f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:35.212224 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34cd29-1ca0-458b-9528-190151dd329f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:35.213157 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5684b12-9d76-49bb-b004-29afd382b988 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:35.223827 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea34cd29-1ca0-458b-9528-190151dd329f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:35.225495 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea34cd29-1ca0-458b-9528-190151dd329f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:35.236803 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:35.241034 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-ea34cd29-1ca0-458b-9528-190151dd329f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:35.241034 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:35.241034 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:35.241034 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:35.241034 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:35.241034 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:35.241034 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:35.241034 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:35.241034 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:35.241034 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:35.241034 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:35.241034 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:35.241034 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:28:35.241034 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:28:35.241034 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:28:35.241034 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:28:35.242328 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:35.242328 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:35.242328 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:35.242328 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:35.242328 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:35.242328 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:35.242328 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:35.242328 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:35.242328 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:35.242328 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:35.242328 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:35.242328 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:35.242328 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:35.242328 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:35.242328 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:35.242328 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:35.242328 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:28:35.243177 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:28:35.243177 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:35.243177 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:35.245684 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 237/466] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:35.254848 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea34cd29-1ca0-458b-9528-190151dd329f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:35.264019 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2def19e-e07d-45a1-b39a-47b99d8d89af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:35.264512 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2def19e-e07d-45a1-b39a-47b99d8d89af None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:35.264826 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2def19e-e07d-45a1-b39a-47b99d8d89af None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:35.265151 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2def19e-e07d-45a1-b39a-47b99d8d89af None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:35.265840 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2def19e-e07d-45a1-b39a-47b99d8d89af None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:35.268840 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2def19e-e07d-45a1-b39a-47b99d8d89af None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:35.304882 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 230/467] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 15:28:35 2022] POST /identity/v3/users => generated 246 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:28:35.306658 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2def19e-e07d-45a1-b39a-47b99d8d89af None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:35.414862 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea34cd29-1ca0-458b-9528-190151dd329f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:35.494045 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9453ad9-4ea5-40a8-8918-bf8041e52e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:35.494475 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9453ad9-4ea5-40a8-8918-bf8041e52e64 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:35.494751 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9453ad9-4ea5-40a8-8918-bf8041e52e64 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:35.495019 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9453ad9-4ea5-40a8-8918-bf8041e52e64 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:35.495756 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9453ad9-4ea5-40a8-8918-bf8041e52e64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:35.496057 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9453ad9-4ea5-40a8-8918-bf8041e52e64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:35.497447 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9453ad9-4ea5-40a8-8918-bf8041e52e64 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:35.512720 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 238/468] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:28:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:35.513898 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9453ad9-4ea5-40a8-8918-bf8041e52e64 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:35.518552 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2def19e-e07d-45a1-b39a-47b99d8d89af None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:35.550426 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:35.558886 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f488ebff-7c8f-4ab8-a3ad-74ab017c20fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:35.559516 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488ebff-7c8f-4ab8-a3ad-74ab017c20fd None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:35.560226 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488ebff-7c8f-4ab8-a3ad-74ab017c20fd None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:35.560506 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488ebff-7c8f-4ab8-a3ad-74ab017c20fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:35.575208 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f488ebff-7c8f-4ab8-a3ad-74ab017c20fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:35.576658 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f488ebff-7c8f-4ab8-a3ad-74ab017c20fd None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:35.591909 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-f488ebff-7c8f-4ab8-a3ad-74ab017c20fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:35.591909 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:35.591909 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:35.591909 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:35.591909 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:35.591909 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:35.591909 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:35.591909 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:35.591909 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:35.591909 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:35.591909 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:35.591909 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:35.591909 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:28:35.591909 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:28:35.591909 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:28:35.591909 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:28:35.593011 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:35.593011 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:35.593011 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:35.593011 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:35.593011 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:35.593011 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:35.593011 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:35.593011 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:35.593011 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:35.593011 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:35.593011 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:35.593011 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:35.593011 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:35.593011 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:35.593011 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:35.593011 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:35.593011 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:28:35.593897 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:28:35.593897 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:35.593897 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:35.593897 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 231/469] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:35.594935 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3c7d2f-40af-475b-b30e-18e9bb1d045a None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:35.594935 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3c7d2f-40af-475b-b30e-18e9bb1d045a None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:35.595099 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3c7d2f-40af-475b-b30e-18e9bb1d045a None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:35.596386 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 232/470] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:35.596914 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3c7d2f-40af-475b-b30e-18e9bb1d045a None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:35.628556 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:35.630430 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9453ad9-4ea5-40a8-8918-bf8041e52e64 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:35.671853 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf16f2ea-4cd9-4027-9c8e-331b9b57b24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:35.672548 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf16f2ea-4cd9-4027-9c8e-331b9b57b24f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:35.672931 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf16f2ea-4cd9-4027-9c8e-331b9b57b24f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:35.673203 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf16f2ea-4cd9-4027-9c8e-331b9b57b24f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:35.684832 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf16f2ea-4cd9-4027-9c8e-331b9b57b24f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:35.686578 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf16f2ea-4cd9-4027-9c8e-331b9b57b24f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:35.702837 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-cf16f2ea-4cd9-4027-9c8e-331b9b57b24f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:35.702837 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:35.702837 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:35.702837 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:35.702837 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:35.702837 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:35.702837 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:35.702837 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:35.702837 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:35.702837 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:35.702837 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:35.702837 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:35.702837 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:28:35.702837 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:28:35.702837 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:28:35.702837 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:28:35.703942 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:35.703942 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:35.703942 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:35.703942 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:35.703942 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:35.703942 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:35.703942 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:35.703942 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:35.703942 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:35.703942 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:35.703942 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:35.703942 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:35.703942 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:35.703942 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:35.703942 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:35.703942 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:35.703942 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:28:35.704920 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:28:35.704920 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:35.704920 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:35.704920 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 239/471] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:35.708370 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3c7d2f-40af-475b-b30e-18e9bb1d045a None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:35.709440 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf16f2ea-4cd9-4027-9c8e-331b9b57b24f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:35.747137 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a2e56-c99e-455a-a9e7-97ed90e5ae2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:35.747734 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a2e56-c99e-455a-a9e7-97ed90e5ae2f None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:35.748003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a2e56-c99e-455a-a9e7-97ed90e5ae2f None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:35.748268 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a2e56-c99e-455a-a9e7-97ed90e5ae2f None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:35.748902 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571a2e56-c99e-455a-a9e7-97ed90e5ae2f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:35.750349 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571a2e56-c99e-455a-a9e7-97ed90e5ae2f None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:35.771937 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 233/472] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 15:28:35 2022] POST /identity/v3/users => generated 248 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:28:35.773128 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e71eb-0416-4ef2-a27b-4b74470b3793 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:35.773407 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e71eb-0416-4ef2-a27b-4b74470b3793 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:35.773676 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e71eb-0416-4ef2-a27b-4b74470b3793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:35.793376 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-db8e71eb-0416-4ef2-a27b-4b74470b3793 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:35.794687 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-db8e71eb-0416-4ef2-a27b-4b74470b3793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:35.820929 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:35.845478 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf16f2ea-4cd9-4027-9c8e-331b9b57b24f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:35.880664 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 234/473] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:35.881731 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8e71eb-0416-4ef2-a27b-4b74470b3793 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:35.899199 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e58563b-96fe-426b-8f94-34abb53991ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:35.900824 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e58563b-96fe-426b-8f94-34abb53991ba None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:35.901285 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e58563b-96fe-426b-8f94-34abb53991ba None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:35.901836 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e58563b-96fe-426b-8f94-34abb53991ba None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:35.902844 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e58563b-96fe-426b-8f94-34abb53991ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:35.903428 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e58563b-96fe-426b-8f94-34abb53991ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:35.905198 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:35.905616 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e58563b-96fe-426b-8f94-34abb53991ba None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:35.920718 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 240/474] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:28:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:35.934695 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e58563b-96fe-426b-8f94-34abb53991ba None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:36.001443 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8e71eb-0416-4ef2-a27b-4b74470b3793 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:36.038291 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4c1a6d-aa10-49fb-8e6e-8ed0f3272107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:36.038890 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c1a6d-aa10-49fb-8e6e-8ed0f3272107 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:36.039161 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c1a6d-aa10-49fb-8e6e-8ed0f3272107 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:36.039430 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c1a6d-aa10-49fb-8e6e-8ed0f3272107 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:36.040062 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4c1a6d-aa10-49fb-8e6e-8ed0f3272107 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:36.040437 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4c1a6d-aa10-49fb-8e6e-8ed0f3272107 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:36.041771 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4c1a6d-aa10-49fb-8e6e-8ed0f3272107 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:36.055020 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 235/475] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:28:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:36.061733 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4c1a6d-aa10-49fb-8e6e-8ed0f3272107 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:36.112878 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e58563b-96fe-426b-8f94-34abb53991ba None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:36.164149 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:36.166064 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ce8d8f-3746-4147-9b41-bda50f7f50be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:36.166781 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce8d8f-3746-4147-9b41-bda50f7f50be None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:36.167171 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce8d8f-3746-4147-9b41-bda50f7f50be None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:36.167854 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce8d8f-3746-4147-9b41-bda50f7f50be None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:36.168849 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ce8d8f-3746-4147-9b41-bda50f7f50be None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:36.171789 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ce8d8f-3746-4147-9b41-bda50f7f50be None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:36.194931 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:36.197342 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4c1a6d-aa10-49fb-8e6e-8ed0f3272107 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:36.204629 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 241/476] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 15:28:35 2022] POST /identity/v3/users => generated 248 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:28:36.238420 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ce8d8f-3746-4147-9b41-bda50f7f50be None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:36.239363 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c50b53d-3e43-47e1-ac47-409520ccdbc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:36.240152 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50b53d-3e43-47e1-ac47-409520ccdbc7 None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:36.240583 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50b53d-3e43-47e1-ac47-409520ccdbc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:36.240993 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50b53d-3e43-47e1-ac47-409520ccdbc7 None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:36.241794 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c50b53d-3e43-47e1-ac47-409520ccdbc7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:36.244440 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c50b53d-3e43-47e1-ac47-409520ccdbc7 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:36.275229 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 236/477] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 15:28:36 2022] POST /identity/v3/users => generated 249 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:28:36.425344 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ce8d8f-3746-4147-9b41-bda50f7f50be None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:36.475894 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1566ab-8b59-44f6-b680-a990f12217c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:36.476510 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1566ab-8b59-44f6-b680-a990f12217c0 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:36.476796 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1566ab-8b59-44f6-b680-a990f12217c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:36.477072 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1566ab-8b59-44f6-b680-a990f12217c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:36.478304 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1566ab-8b59-44f6-b680-a990f12217c0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:36.479958 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1566ab-8b59-44f6-b680-a990f12217c0 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:36.495374 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.sql.core [None req-1e1566ab-8b59-44f6-b680-a990f12217c0 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 28 15:28:36.495374 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 28 15:28:36.495374 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [parameters: {'id': 'd745cd09ecc34a088d2ae162d66e9ad0', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 28 15:28:36.495374 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81605) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 28 15:28:36.512035 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-1e1566ab-8b59-44f6-b680-a990f12217c0 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 28 15:28:36.512035 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:36.512035 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 28 15:28:36.512035 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 28 15:28:36.512035 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 28 15:28:36.512035 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 28 15:28:36.512035 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 28 15:28:36.512035 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application result = self._query(query) Sep 28 15:28:36.512035 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 28 15:28:36.512035 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application conn.query(q) Sep 28 15:28:36.512035 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 28 15:28:36.512035 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 28 15:28:36.512035 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 28 15:28:36.512035 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application result.read() Sep 28 15:28:36.512035 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 28 15:28:36.512981 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 28 15:28:36.512981 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 28 15:28:36.512981 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 28 15:28:36.512981 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 28 15:28:36.512981 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 28 15:28:36.512981 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 28 15:28:36.512981 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 28 15:28:36.512981 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 28 15:28:36.512981 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:36.512981 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 28 15:28:36.512981 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:36.512981 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:36.512981 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 28 15:28:36.512981 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 28 15:28:36.512981 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 28 15:28:36.512981 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return ref.to_dict() Sep 28 15:28:36.512981 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 28 15:28:36.512981 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application next(self.gen) Sep 28 15:28:36.513933 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 28 15:28:36.513933 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application yield resource Sep 28 15:28:36.513933 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 28 15:28:36.513933 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application next(self.gen) Sep 28 15:28:36.513933 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 28 15:28:36.513933 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self.session.rollback() Sep 28 15:28:36.513933 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 28 15:28:36.513933 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self.force_reraise() Sep 28 15:28:36.513933 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 28 15:28:36.513933 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise self.value Sep 28 15:28:36.513933 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 28 15:28:36.513933 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 28 15:28:36.513933 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 28 15:28:36.513933 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application session.commit() Sep 28 15:28:36.513933 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 28 15:28:36.513933 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 28 15:28:36.513933 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 28 15:28:36.514796 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._prepare_impl() Sep 28 15:28:36.514796 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 28 15:28:36.514796 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self.session.flush() Sep 28 15:28:36.514796 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 28 15:28:36.514796 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._flush(objects) Sep 28 15:28:36.514796 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 28 15:28:36.514796 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 28 15:28:36.514796 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 28 15:28:36.514796 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application compat.raise_( Sep 28 15:28:36.514796 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 28 15:28:36.514796 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception Sep 28 15:28:36.514796 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 28 15:28:36.514796 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application flush_context.execute() Sep 28 15:28:36.514796 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 28 15:28:36.514796 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rec.execute(self) Sep 28 15:28:36.514796 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 28 15:28:36.514796 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 28 15:28:36.515766 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 28 15:28:36.515766 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 28 15:28:36.515766 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 28 15:28:36.515766 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 28 15:28:36.515766 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 28 15:28:36.515766 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 28 15:28:36.515766 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 28 15:28:36.515766 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 28 15:28:36.515766 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 28 15:28:36.515766 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 28 15:28:36.515766 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 28 15:28:36.515766 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 28 15:28:36.515766 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 28 15:28:36.515766 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 28 15:28:36.515766 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 28 15:28:36.515766 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception Sep 28 15:28:36.516597 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 28 15:28:36.516597 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 28 15:28:36.516597 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 28 15:28:36.516597 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 28 15:28:36.516597 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 28 15:28:36.516597 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application result = self._query(query) Sep 28 15:28:36.516597 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 28 15:28:36.516597 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application conn.query(q) Sep 28 15:28:36.516597 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 28 15:28:36.516597 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 28 15:28:36.516597 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 28 15:28:36.516597 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application result.read() Sep 28 15:28:36.516597 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 28 15:28:36.516597 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 28 15:28:36.516597 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 28 15:28:36.516597 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 28 15:28:36.517373 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 28 15:28:36.517373 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 28 15:28:36.517373 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 28 15:28:36.517373 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 28 15:28:36.517373 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 28 15:28:36.517373 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 28 15:28:36.517373 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [parameters: {'id': 'd745cd09ecc34a088d2ae162d66e9ad0', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 28 15:28:36.517373 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 28 15:28:36.517373 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:36.517373 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:28:36.517373 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:36.517373 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:36.517373 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:36.517373 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:36.517373 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:36.517373 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:36.519975 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:36.519975 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:36.519975 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:36.519975 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:36.519975 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:36.519975 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:36.519975 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 28 15:28:36.519975 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 28 15:28:36.519975 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:36.519975 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:36.519975 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 28 15:28:36.519975 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 28 15:28:36.519975 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 28 15:28:36.519975 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 28 15:28:36.519975 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 28 15:28:36.519975 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:36.522182 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 242/478] 10.222.0.52 () {60 vars in 1131 bytes} [Wed Sep 28 15:28:36 2022] POST /identity/v3/roles => generated 144 bytes in 276 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:36.522318 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c50b53d-3e43-47e1-ac47-409520ccdbc7 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:36.633300 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c50b53d-3e43-47e1-ac47-409520ccdbc7 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:36.664626 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:36.672231 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd0f2c9-b51f-4385-a194-eb692cfea8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:36.673152 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0f2c9-b51f-4385-a194-eb692cfea8be None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:36.673643 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0f2c9-b51f-4385-a194-eb692cfea8be None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:36.674140 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0f2c9-b51f-4385-a194-eb692cfea8be None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:36.688565 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd0f2c9-b51f-4385-a194-eb692cfea8be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:36.694903 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-edd0f2c9-b51f-4385-a194-eb692cfea8be None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:28:36.694903 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:36.694903 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:36.694903 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:36.694903 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:36.694903 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:36.694903 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:36.694903 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:36.694903 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:36.694903 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:36.694903 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:36.694903 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:36.694903 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:36.694903 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:36.694903 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:36.694903 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:36.695787 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:36.695787 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:36.695787 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:36.695787 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:36.695787 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:36.695787 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:36.695787 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:36.695787 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:36.695787 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:36.695787 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:36.695787 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:36.695787 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:36.695787 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:36.695787 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:36.695787 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:36.695787 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:36.697283 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:36.697283 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:36.697283 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:36.697283 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:36.697283 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:36.697283 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:36.697283 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:36.697283 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:36.697283 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:36.697283 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:36.697283 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 28 15:28:36.697283 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:36.699993 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 237/479] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 15:28:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:36.705498 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1566ab-8b59-44f6-b680-a990f12217c0 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:36.772193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:36.813618 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1566ab-8b59-44f6-b680-a990f12217c0 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:36.852431 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d38ed05-ff6f-457a-8998-d0951e228755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:36.853081 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d38ed05-ff6f-457a-8998-d0951e228755 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:36.853359 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d38ed05-ff6f-457a-8998-d0951e228755 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:36.853633 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d38ed05-ff6f-457a-8998-d0951e228755 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:36.854323 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d38ed05-ff6f-457a-8998-d0951e228755 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:36.854627 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d38ed05-ff6f-457a-8998-d0951e228755 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:36.855999 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d38ed05-ff6f-457a-8998-d0951e228755 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:36.864542 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 243/480] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:28:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:28:37.749360 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efdb8d6-9066-4fd2-bf0f-d85f20f5e7f3 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:37.750223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efdb8d6-9066-4fd2-bf0f-d85f20f5e7f3 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:37.750670 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efdb8d6-9066-4fd2-bf0f-d85f20f5e7f3 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:37.752391 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 238/481] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:37.757659 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d9301-fecd-495c-9d3a-4b3aa5d9000b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:37.758407 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d9301-fecd-495c-9d3a-4b3aa5d9000b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:37.758845 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d9301-fecd-495c-9d3a-4b3aa5d9000b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:37.781508 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-e87d9301-fecd-495c-9d3a-4b3aa5d9000b None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:37.782911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e87d9301-fecd-495c-9d3a-4b3aa5d9000b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:37.867914 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 244/482] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:38.320624 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efdb8d6-9066-4fd2-bf0f-d85f20f5e7f3 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:38.450662 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efdb8d6-9066-4fd2-bf0f-d85f20f5e7f3 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:38.501211 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbefb5e-ea4b-48ba-ab24-7eef208b7141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:38.502139 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbefb5e-ea4b-48ba-ab24-7eef208b7141 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:38.502617 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbefb5e-ea4b-48ba-ab24-7eef208b7141 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:38.503030 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbefb5e-ea4b-48ba-ab24-7eef208b7141 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:38.519989 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbefb5e-ea4b-48ba-ab24-7eef208b7141 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:38.525089 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-2cbefb5e-ea4b-48ba-ab24-7eef208b7141 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:28:38.525089 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:38.525089 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:38.525089 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:38.525089 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:38.525089 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:38.525089 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:38.525089 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:38.525089 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:38.525089 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:38.525089 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:38.525089 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:38.525089 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:38.525089 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:38.525089 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:38.525089 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:38.526173 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:38.526173 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:38.526173 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:38.526173 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:38.526173 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:38.526173 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:38.526173 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:38.526173 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:38.526173 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:38.526173 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:38.526173 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:38.526173 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:38.526173 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:38.526173 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:38.526173 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:38.526173 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:38.527172 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:38.527172 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:38.527172 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:38.527172 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:38.527172 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:38.527172 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:38.527172 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:38.527172 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:38.527172 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:38.527172 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:38.527172 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:28:38.527172 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:38.529413 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 239/483] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 15:28:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:38.535361 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87d9301-fecd-495c-9d3a-4b3aa5d9000b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:38.595494 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:38.614260 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a760b65f-e0f6-4f32-aa11-2dcf10af9b6b None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:38.614581 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a760b65f-e0f6-4f32-aa11-2dcf10af9b6b None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:38.614804 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a760b65f-e0f6-4f32-aa11-2dcf10af9b6b None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:38.616255 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 240/484] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:28:38.623234 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2739d059-5a26-4cb0-a886-d56733f06874 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:38.623517 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2739d059-5a26-4cb0-a886-d56733f06874 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:38.623795 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2739d059-5a26-4cb0-a886-d56733f06874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:38.658454 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-2739d059-5a26-4cb0-a886-d56733f06874 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:38.659657 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-2739d059-5a26-4cb0-a886-d56733f06874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:38.665888 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87d9301-fecd-495c-9d3a-4b3aa5d9000b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:38.709491 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10340c40-8bb3-4bcc-b02d-9438c625be16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:38.710481 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10340c40-8bb3-4bcc-b02d-9438c625be16 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:38.710904 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10340c40-8bb3-4bcc-b02d-9438c625be16 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:38.711332 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10340c40-8bb3-4bcc-b02d-9438c625be16 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:38.712695 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10340c40-8bb3-4bcc-b02d-9438c625be16 None admin] RBAC: Adding query filter params (name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:38.713140 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10340c40-8bb3-4bcc-b02d-9438c625be16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:38.714799 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10340c40-8bb3-4bcc-b02d-9438c625be16 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:38.733111 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 245/485] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 15:28:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:38.734809 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:38.738978 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10340c40-8bb3-4bcc-b02d-9438c625be16 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:38.748188 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 241/486] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:38.881564 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10340c40-8bb3-4bcc-b02d-9438c625be16 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:38.906299 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:38.925439 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d78675-597e-4321-b7a1-3e5283430f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:38.926099 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d78675-597e-4321-b7a1-3e5283430f3d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:38.926381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d78675-597e-4321-b7a1-3e5283430f3d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:38.926655 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d78675-597e-4321-b7a1-3e5283430f3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:38.938479 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d78675-597e-4321-b7a1-3e5283430f3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:38.939962 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d78675-597e-4321-b7a1-3e5283430f3d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:38.956756 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-70d78675-597e-4321-b7a1-3e5283430f3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:38.956756 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:38.956756 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:38.956756 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:38.956756 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:38.956756 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:38.956756 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:38.956756 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:38.956756 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:38.956756 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:38.956756 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:38.956756 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:38.956756 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:28:38.956756 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:28:38.956756 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:28:38.956756 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:28:38.957742 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:38.957742 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:38.957742 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:38.957742 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:38.957742 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:38.957742 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:38.957742 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:38.957742 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:38.957742 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:38.957742 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:38.957742 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:38.957742 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:38.957742 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:38.957742 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:38.957742 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:38.957742 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:38.957742 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:28:38.958624 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:28:38.958624 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:38.958624 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:38.958624 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 246/487] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:28:38.966167 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2739d059-5a26-4cb0-a886-d56733f06874 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:39.029354 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13115c9d-a4c6-4956-a0c0-0ef24c5610ee None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:39.029670 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13115c9d-a4c6-4956-a0c0-0ef24c5610ee None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:39.029943 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13115c9d-a4c6-4956-a0c0-0ef24c5610ee None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:39.031702 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 247/488] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:39.036882 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ae67f-48cd-44eb-8854-2e55990ff206 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:39.037165 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ae67f-48cd-44eb-8854-2e55990ff206 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:39.037428 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ae67f-48cd-44eb-8854-2e55990ff206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:39.057424 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-c21ae67f-48cd-44eb-8854-2e55990ff206 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:39.058740 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c21ae67f-48cd-44eb-8854-2e55990ff206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:39.083726 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2739d059-5a26-4cb0-a886-d56733f06874 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:39.120402 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f9e51d-7e49-492f-9cd4-8e44340f4667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:39.121092 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f9e51d-7e49-492f-9cd4-8e44340f4667 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:39.121367 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f9e51d-7e49-492f-9cd4-8e44340f4667 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:39.121628 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f9e51d-7e49-492f-9cd4-8e44340f4667 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:39.122286 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f9e51d-7e49-492f-9cd4-8e44340f4667 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:39.122581 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f9e51d-7e49-492f-9cd4-8e44340f4667 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:39.124149 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f9e51d-7e49-492f-9cd4-8e44340f4667 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:39.131995 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 248/489] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:39.133290 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c69a9-8541-4696-ad21-2ca88b407e97 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:39.133575 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c69a9-8541-4696-ad21-2ca88b407e97 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:39.136265 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 242/490] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:28:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:39.137916 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c69a9-8541-4696-ad21-2ca88b407e97 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:39.139473 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 249/491] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:39.144981 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb728a3b-acd9-4e3e-95dd-d8bcbd9885d0 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:39.145259 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb728a3b-acd9-4e3e-95dd-d8bcbd9885d0 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:39.145528 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb728a3b-acd9-4e3e-95dd-d8bcbd9885d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:39.146698 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9c69a9-8541-4696-ad21-2ca88b407e97 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:39.170467 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-bb728a3b-acd9-4e3e-95dd-d8bcbd9885d0 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:39.171812 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-bb728a3b-acd9-4e3e-95dd-d8bcbd9885d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:39.215598 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:39.245587 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 243/492] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:39.250744 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb728a3b-acd9-4e3e-95dd-d8bcbd9885d0 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:39.330797 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:39.335202 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9c69a9-8541-4696-ad21-2ca88b407e97 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:39.377993 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb728a3b-acd9-4e3e-95dd-d8bcbd9885d0 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:39.378170 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faced4cc-3d22-45f4-ab27-e92b2660f3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:39.378646 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faced4cc-3d22-45f4-ab27-e92b2660f3bc None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:39.378925 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faced4cc-3d22-45f4-ab27-e92b2660f3bc None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:39.379194 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faced4cc-3d22-45f4-ab27-e92b2660f3bc None admin] PATH_INFO: `/v3/users/nova` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:39.388856 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faced4cc-3d22-45f4-ab27-e92b2660f3bc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:39.390669 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faced4cc-3d22-45f4-ab27-e92b2660f3bc None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:39.407238 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-faced4cc-3d22-45f4-ab27-e92b2660f3bc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 28 15:28:39.407238 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:39.407238 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:39.407238 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:39.407238 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:39.407238 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:39.407238 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:39.407238 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:39.407238 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:39.407238 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:39.407238 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:39.407238 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:39.407238 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:28:39.407238 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:28:39.407238 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:28:39.407238 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:28:39.408201 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:39.408201 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:39.408201 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:28:39.408201 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:28:39.408201 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:28:39.408201 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:28:39.408201 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:39.408201 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:39.408201 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:39.408201 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:39.408201 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:39.408201 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:39.408201 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:39.408201 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:39.408201 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:39.408201 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:39.408201 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:39.408201 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:39.409677 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:39.409677 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:39.409677 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:28:39.409677 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:28:39.409677 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:28:39.409677 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:28:39.409677 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:28:39.409677 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:28:39.409677 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 28 15:28:39.409677 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:39.409677 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 250/493] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Sep 28 15:28:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:39.414066 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faced4cc-3d22-45f4-ab27-e92b2660f3bc None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:39.430483 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7859a832-6ccc-492c-a0bf-423f8568b77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:39.432004 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7859a832-6ccc-492c-a0bf-423f8568b77a None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:39.432482 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7859a832-6ccc-492c-a0bf-423f8568b77a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:39.432947 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7859a832-6ccc-492c-a0bf-423f8568b77a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:39.450463 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7859a832-6ccc-492c-a0bf-423f8568b77a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:39.455589 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-7859a832-6ccc-492c-a0bf-423f8568b77a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:28:39.455589 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:39.455589 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:39.455589 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:39.455589 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:39.455589 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:39.455589 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:39.455589 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:39.455589 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:39.455589 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:39.455589 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:39.455589 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:39.455589 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:39.455589 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:39.455589 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:39.455589 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:39.456481 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:39.456481 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:39.456481 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:39.456481 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:39.456481 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:39.456481 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:39.456481 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:39.456481 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:39.456481 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:39.456481 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:39.456481 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:39.456481 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:39.456481 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:39.456481 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:39.456481 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:39.456481 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:39.457286 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:39.457286 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:39.457286 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:39.457286 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:39.457286 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:39.457286 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:39.457286 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:39.457286 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:39.457286 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:39.457286 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:39.457286 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:28:39.457286 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:39.464200 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 244/494] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 15:28:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:28:39.465256 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7859a832-6ccc-492c-a0bf-423f8568b77a None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:39.501179 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:39.535881 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faced4cc-3d22-45f4-ab27-e92b2660f3bc None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:39.575302 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d81be1-7c06-45d8-bbfe-1312ba525468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:39.575913 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d81be1-7c06-45d8-bbfe-1312ba525468 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:39.576192 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d81be1-7c06-45d8-bbfe-1312ba525468 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:39.576461 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d81be1-7c06-45d8-bbfe-1312ba525468 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:39.577187 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d81be1-7c06-45d8-bbfe-1312ba525468 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:39.577489 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d81be1-7c06-45d8-bbfe-1312ba525468 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:39.578976 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d81be1-7c06-45d8-bbfe-1312ba525468 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:39.608420 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7859a832-6ccc-492c-a0bf-423f8568b77a None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:39.626600 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 251/495] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Sep 28 15:28:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:39.627288 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d81be1-7c06-45d8-bbfe-1312ba525468 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:39.645817 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:39.650485 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ab0bf7-3060-4b88-94ac-13fcce272063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:39.651271 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab0bf7-3060-4b88-94ac-13fcce272063 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:39.651563 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab0bf7-3060-4b88-94ac-13fcce272063 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:39.651831 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ab0bf7-3060-4b88-94ac-13fcce272063 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:39.664863 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ab0bf7-3060-4b88-94ac-13fcce272063 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:39.670234 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-c6ab0bf7-3060-4b88-94ac-13fcce272063 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:28:39.670234 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:39.670234 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:39.670234 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:39.670234 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:39.670234 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:39.670234 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:39.670234 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:39.670234 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:39.670234 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:39.670234 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:39.670234 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:39.670234 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:39.670234 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:39.670234 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:39.670234 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:39.672293 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:39.672293 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:39.672293 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:39.672293 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:39.672293 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:39.672293 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:39.672293 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:39.672293 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:39.672293 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:39.672293 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:39.672293 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:39.672293 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:39.672293 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:39.672293 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:39.672293 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:39.672293 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:39.673249 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:39.673249 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:39.673249 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:39.673249 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:39.673249 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:39.673249 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:39.673249 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:39.673249 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:39.673249 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:39.673249 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:39.673249 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:28:39.673249 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:39.673249 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 245/496] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 15:28:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:39.673249 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ab0bf7-3060-4b88-94ac-13fcce272063 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:39.737110 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d81be1-7c06-45d8-bbfe-1312ba525468 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:39.771745 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:39.773786 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd210a7-668e-46ec-85c8-cfa3ef16748d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:39.774933 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd210a7-668e-46ec-85c8-cfa3ef16748d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:39.775279 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd210a7-668e-46ec-85c8-cfa3ef16748d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:39.775498 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd210a7-668e-46ec-85c8-cfa3ef16748d None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:39.776209 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd210a7-668e-46ec-85c8-cfa3ef16748d None admin] RBAC: Adding query filter params (name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:39.776477 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd210a7-668e-46ec-85c8-cfa3ef16748d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:39.777885 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd210a7-668e-46ec-85c8-cfa3ef16748d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:39.782385 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ab0bf7-3060-4b88-94ac-13fcce272063 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:39.787093 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 252/497] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 15:28:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:39.788114 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd210a7-668e-46ec-85c8-cfa3ef16748d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:39.819715 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef3b962-e04c-4f8d-b24e-1fab310bb691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:39.820334 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef3b962-e04c-4f8d-b24e-1fab310bb691 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:39.820669 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef3b962-e04c-4f8d-b24e-1fab310bb691 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:39.820872 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef3b962-e04c-4f8d-b24e-1fab310bb691 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:39.831907 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef3b962-e04c-4f8d-b24e-1fab310bb691 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:39.836637 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-5ef3b962-e04c-4f8d-b24e-1fab310bb691 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:28:39.836637 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:39.836637 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:39.836637 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:39.836637 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:39.836637 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:39.836637 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:39.836637 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:39.836637 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:39.836637 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:39.836637 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:39.836637 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:39.836637 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:39.836637 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:39.836637 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:39.836637 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:39.838163 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:39.838163 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:39.838163 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:39.838163 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:39.838163 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:39.838163 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:39.838163 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:39.838163 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:39.838163 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:39.838163 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:39.838163 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:39.838163 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:39.838163 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:39.838163 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:39.838163 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:39.838163 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:39.838986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:39.838986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:39.838986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:39.838986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:39.838986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:39.838986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:39.838986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:39.838986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:39.838986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:39.838986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:39.838986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:28:39.838986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:39.838986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 246/498] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 15:28:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:39.838986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef3b962-e04c-4f8d-b24e-1fab310bb691 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:39.893114 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:39.916371 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd210a7-668e-46ec-85c8-cfa3ef16748d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:39.949411 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef3b962-e04c-4f8d-b24e-1fab310bb691 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:39.958624 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae30f741-b827-4418-9888-5645bbf6bd17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:39.959245 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30f741-b827-4418-9888-5645bbf6bd17 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:39.959532 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30f741-b827-4418-9888-5645bbf6bd17 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:39.959804 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae30f741-b827-4418-9888-5645bbf6bd17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:39.988277 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb9b792-70ee-4db7-88e6-72811eb82597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:39.988927 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb9b792-70ee-4db7-88e6-72811eb82597 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:39.989211 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb9b792-70ee-4db7-88e6-72811eb82597 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:39.989481 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb9b792-70ee-4db7-88e6-72811eb82597 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:39.990288 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb9b792-70ee-4db7-88e6-72811eb82597 None admin] RBAC: Adding query filter params (name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:39.990595 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb9b792-70ee-4db7-88e6-72811eb82597 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:39.992071 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb9b792-70ee-4db7-88e6-72811eb82597 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:40.005877 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 247/499] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 15:28:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:40.005877 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb9b792-70ee-4db7-88e6-72811eb82597 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:40.020466 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:40.036823 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae30f741-b827-4418-9888-5645bbf6bd17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:40.038333 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae30f741-b827-4418-9888-5645bbf6bd17 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:40.108395 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb9b792-70ee-4db7-88e6-72811eb82597 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:40.115954 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:40.136429 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 253/500] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:28:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:28:40.137236 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae30f741-b827-4418-9888-5645bbf6bd17 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:40.142142 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af9e31c-2171-44a1-9466-e7f0b02e3904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:40.142767 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af9e31c-2171-44a1-9466-e7f0b02e3904 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:40.143047 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af9e31c-2171-44a1-9466-e7f0b02e3904 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:40.143317 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af9e31c-2171-44a1-9466-e7f0b02e3904 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:40.154963 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af9e31c-2171-44a1-9466-e7f0b02e3904 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:40.156446 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af9e31c-2171-44a1-9466-e7f0b02e3904 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:40.170881 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-7af9e31c-2171-44a1-9466-e7f0b02e3904 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:40.170881 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:40.170881 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:40.170881 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:40.170881 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:40.170881 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:40.170881 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:40.170881 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:40.170881 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:40.170881 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:40.170881 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:40.170881 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:40.170881 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:28:40.170881 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:28:40.170881 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:28:40.170881 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:28:40.171970 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:40.171970 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:40.171970 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:40.171970 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:40.171970 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:40.171970 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:40.171970 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:40.171970 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:40.171970 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:40.171970 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:40.171970 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:40.171970 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:40.171970 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:40.171970 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:40.171970 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:40.171970 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:40.171970 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:28:40.172837 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:28:40.172837 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:40.172837 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:40.172837 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 248/501] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:40.173047 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af9e31c-2171-44a1-9466-e7f0b02e3904 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:40.239133 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:40.239578 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae30f741-b827-4418-9888-5645bbf6bd17 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:40.273422 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089008f3-917f-41e0-88bf-267dad0a5be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:40.274058 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089008f3-917f-41e0-88bf-267dad0a5be1 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:40.274336 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089008f3-917f-41e0-88bf-267dad0a5be1 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:40.275167 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089008f3-917f-41e0-88bf-267dad0a5be1 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:40.275824 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089008f3-917f-41e0-88bf-267dad0a5be1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:40.276117 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089008f3-917f-41e0-88bf-267dad0a5be1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:40.277496 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089008f3-917f-41e0-88bf-267dad0a5be1 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:40.277587 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af9e31c-2171-44a1-9466-e7f0b02e3904 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:40.285471 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 254/502] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 15:28:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:40.286671 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d4e369-e8e4-4631-bfc1-2c495d0d014c None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:40.286944 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d4e369-e8e4-4631-bfc1-2c495d0d014c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:40.287215 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d4e369-e8e4-4631-bfc1-2c495d0d014c None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:40.288374 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 255/503] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:40.289052 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d4e369-e8e4-4631-bfc1-2c495d0d014c None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:40.317991 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d45ff1-7c19-4021-9e77-38fc412c3ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:40.318606 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d45ff1-7c19-4021-9e77-38fc412c3ed0 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:40.318877 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d45ff1-7c19-4021-9e77-38fc412c3ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:40.319141 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d45ff1-7c19-4021-9e77-38fc412c3ed0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:40.329904 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d45ff1-7c19-4021-9e77-38fc412c3ed0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:40.331324 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d45ff1-7c19-4021-9e77-38fc412c3ed0 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:40.333622 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:40.345838 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-f6d45ff1-7c19-4021-9e77-38fc412c3ed0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:40.345838 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:40.345838 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:40.345838 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:40.345838 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:40.345838 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:40.345838 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:40.345838 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:40.345838 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:40.345838 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:40.345838 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:40.345838 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:40.345838 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:28:40.345838 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:28:40.345838 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:28:40.345838 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:28:40.346766 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:40.346766 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:40.346766 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:40.346766 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:40.346766 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:40.346766 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:40.346766 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:40.346766 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:40.346766 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:40.346766 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:40.346766 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:40.346766 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:40.346766 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:40.346766 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:40.346766 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:40.346766 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:40.346766 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:28:40.347599 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:28:40.347599 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:40.347599 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:40.347599 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 249/504] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:40.347819 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d45ff1-7c19-4021-9e77-38fc412c3ed0 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:40.396064 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d4e369-e8e4-4631-bfc1-2c495d0d014c None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:40.428500 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74dfa0f-fa57-48fe-a853-9b0605585bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:40.429051 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74dfa0f-fa57-48fe-a853-9b0605585bba None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:40.429331 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74dfa0f-fa57-48fe-a853-9b0605585bba None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:40.429603 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74dfa0f-fa57-48fe-a853-9b0605585bba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:40.440925 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74dfa0f-fa57-48fe-a853-9b0605585bba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:40.442471 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74dfa0f-fa57-48fe-a853-9b0605585bba None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:40.446680 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d45ff1-7c19-4021-9e77-38fc412c3ed0 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:40.456253 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-e74dfa0f-fa57-48fe-a853-9b0605585bba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:40.456253 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:40.456253 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:40.456253 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:40.456253 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:40.456253 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:40.456253 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:40.456253 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:40.456253 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:40.456253 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:40.456253 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:40.456253 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:40.456253 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:28:40.456253 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:28:40.456253 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:28:40.456253 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:28:40.457121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:40.457121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:40.457121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:40.457121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:40.457121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:40.457121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:40.457121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:40.457121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:40.457121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:40.457121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:40.457121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:40.457121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:40.457121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:40.457121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:40.457121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:40.457121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:40.457121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:28:40.458604 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:28:40.458604 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:40.458604 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:40.458604 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 256/505] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:40.458604 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74dfa0f-fa57-48fe-a853-9b0605585bba None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:40.468166 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:40.487886 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec39842-0505-40d1-8bb7-3d182b66f98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:40.488488 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec39842-0505-40d1-8bb7-3d182b66f98a None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:40.488769 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec39842-0505-40d1-8bb7-3d182b66f98a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:40.489029 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec39842-0505-40d1-8bb7-3d182b66f98a None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:40.489672 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec39842-0505-40d1-8bb7-3d182b66f98a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:40.490002 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec39842-0505-40d1-8bb7-3d182b66f98a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:40.491358 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec39842-0505-40d1-8bb7-3d182b66f98a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:40.503643 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 250/506] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:28:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:40.504766 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d7941-c319-4a2b-901c-4b5721feb521 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:40.505042 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d7941-c319-4a2b-901c-4b5721feb521 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:40.505308 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d7941-c319-4a2b-901c-4b5721feb521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:40.522652 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-c10d7941-c319-4a2b-901c-4b5721feb521 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:40.523837 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c10d7941-c319-4a2b-901c-4b5721feb521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:40.556453 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:40.563841 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74dfa0f-fa57-48fe-a853-9b0605585bba None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:40.589637 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 251/507] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:40.590639 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10d7941-c319-4a2b-901c-4b5721feb521 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:40.601645 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c50fef-b0e8-45a5-a0b6-93d44c2b3326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:40.602305 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c50fef-b0e8-45a5-a0b6-93d44c2b3326 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:40.603185 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c50fef-b0e8-45a5-a0b6-93d44c2b3326 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:40.603437 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c50fef-b0e8-45a5-a0b6-93d44c2b3326 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:40.614549 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c50fef-b0e8-45a5-a0b6-93d44c2b3326 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:40.616067 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c50fef-b0e8-45a5-a0b6-93d44c2b3326 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:40.631267 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-24c50fef-b0e8-45a5-a0b6-93d44c2b3326 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:40.631267 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:40.631267 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:40.631267 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:40.631267 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:40.631267 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:40.631267 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:40.631267 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:40.631267 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:40.631267 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:40.631267 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:40.631267 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:40.631267 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:28:40.631267 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:28:40.631267 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:28:40.631267 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:28:40.632150 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:40.632150 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:40.632150 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:40.632150 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:40.632150 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:40.632150 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:40.632150 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:40.632150 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:40.632150 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:40.632150 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:40.632150 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:40.632150 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:40.632150 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:40.632150 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:40.632150 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:40.632150 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:40.632150 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:28:40.633044 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:28:40.633044 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:40.633044 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:40.633044 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 257/508] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:40.633266 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c50fef-b0e8-45a5-a0b6-93d44c2b3326 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:40.695378 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10d7941-c319-4a2b-901c-4b5721feb521 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:40.706091 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:40.733529 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a116697-59be-454e-8000-689b8dc50bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:40.734155 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a116697-59be-454e-8000-689b8dc50bed None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:40.734257 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c50fef-b0e8-45a5-a0b6-93d44c2b3326 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:40.734441 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a116697-59be-454e-8000-689b8dc50bed None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:40.734688 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a116697-59be-454e-8000-689b8dc50bed None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:40.735312 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a116697-59be-454e-8000-689b8dc50bed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:40.735862 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a116697-59be-454e-8000-689b8dc50bed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:40.737346 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a116697-59be-454e-8000-689b8dc50bed None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:40.753269 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 252/509] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:28:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:40.753269 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a116697-59be-454e-8000-689b8dc50bed None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:40.772953 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c52fa18-f1e2-4244-9225-8d0c0c1ea79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:40.773554 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c52fa18-f1e2-4244-9225-8d0c0c1ea79a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:40.773828 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c52fa18-f1e2-4244-9225-8d0c0c1ea79a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:40.775033 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c52fa18-f1e2-4244-9225-8d0c0c1ea79a None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:40.775668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c52fa18-f1e2-4244-9225-8d0c0c1ea79a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:40.776242 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c52fa18-f1e2-4244-9225-8d0c0c1ea79a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:40.777778 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c52fa18-f1e2-4244-9225-8d0c0c1ea79a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:40.794022 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 258/510] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:28:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:40.795190 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c52fa18-f1e2-4244-9225-8d0c0c1ea79a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:40.806883 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:40.860106 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a116697-59be-454e-8000-689b8dc50bed None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:40.893666 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac3a99-a956-4ca4-9e73-707771c81136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:40.894305 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac3a99-a956-4ca4-9e73-707771c81136 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:40.894572 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac3a99-a956-4ca4-9e73-707771c81136 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:40.894836 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac3a99-a956-4ca4-9e73-707771c81136 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:40.899474 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c52fa18-f1e2-4244-9225-8d0c0c1ea79a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:40.902013 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ac3a99-a956-4ca4-9e73-707771c81136 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:40.903434 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ac3a99-a956-4ca4-9e73-707771c81136 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:40.915959 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-e1ac3a99-a956-4ca4-9e73-707771c81136 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 28 15:28:40.915959 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:40.915959 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:40.915959 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:40.915959 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:40.915959 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:40.915959 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:40.915959 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:40.915959 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:40.915959 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:40.915959 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:40.915959 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:40.915959 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:28:40.915959 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:28:40.915959 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:28:40.915959 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:28:40.916866 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:40.916866 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:40.916866 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:28:40.916866 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:28:40.916866 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:28:40.916866 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:28:40.916866 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:40.916866 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:40.916866 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:40.916866 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:40.916866 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:40.916866 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:40.916866 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:40.916866 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:40.916866 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:40.916866 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:40.916866 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:40.916866 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:40.917765 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:40.917765 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:40.917765 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:28:40.917765 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:28:40.917765 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:28:40.917765 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:28:40.917765 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:28:40.917765 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:28:40.917765 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 28 15:28:40.917765 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:40.917765 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 253/511] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:28:40 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:40.917765 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac3a99-a956-4ca4-9e73-707771c81136 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:40.934860 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f901c0c7-6726-4b9d-8ada-5f7ea8970fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:40.935472 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901c0c7-6726-4b9d-8ada-5f7ea8970fd6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:40.935755 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901c0c7-6726-4b9d-8ada-5f7ea8970fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:40.936031 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901c0c7-6726-4b9d-8ada-5f7ea8970fd6 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:40.936665 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f901c0c7-6726-4b9d-8ada-5f7ea8970fd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:40.936969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f901c0c7-6726-4b9d-8ada-5f7ea8970fd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:40.938387 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f901c0c7-6726-4b9d-8ada-5f7ea8970fd6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:40.948416 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:40.953745 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 259/512] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:28:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:40.953745 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f901c0c7-6726-4b9d-8ada-5f7ea8970fd6 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:41.014184 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:41.018721 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac3a99-a956-4ca4-9e73-707771c81136 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:41.051166 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ffd19b-9ce4-4f4b-a8c4-eb16cbe5b27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:41.051781 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ffd19b-9ce4-4f4b-a8c4-eb16cbe5b27d None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:41.051874 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f901c0c7-6726-4b9d-8ada-5f7ea8970fd6 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:41.052076 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ffd19b-9ce4-4f4b-a8c4-eb16cbe5b27d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:41.052341 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ffd19b-9ce4-4f4b-a8c4-eb16cbe5b27d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:41.060151 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ffd19b-9ce4-4f4b-a8c4-eb16cbe5b27d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:41.061588 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ffd19b-9ce4-4f4b-a8c4-eb16cbe5b27d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:41.072411 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-f6ffd19b-9ce4-4f4b-a8c4-eb16cbe5b27d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 28 15:28:41.072411 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:41.072411 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:41.072411 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:41.072411 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:41.072411 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:41.072411 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:41.072411 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:41.072411 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:41.072411 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:41.072411 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:41.072411 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:41.072411 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:28:41.072411 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:28:41.072411 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:28:41.072411 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:28:41.073276 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:41.073276 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:41.073276 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:28:41.073276 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:28:41.073276 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:28:41.073276 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:28:41.073276 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:41.073276 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:41.073276 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:41.073276 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:41.073276 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:41.073276 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:41.073276 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:41.073276 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:41.073276 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:41.073276 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:41.073276 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:41.073276 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:41.074189 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:41.074189 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:41.074189 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:28:41.074189 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:28:41.074189 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:28:41.074189 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:28:41.074189 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:28:41.074189 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:28:41.074189 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 28 15:28:41.074189 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:41.074189 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 254/513] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:28:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:41.074742 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ffd19b-9ce4-4f4b-a8c4-eb16cbe5b27d None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:41.088417 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc1f38-ef66-4d82-a591-9fd1b6d72248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:41.089036 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc1f38-ef66-4d82-a591-9fd1b6d72248 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:41.089311 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc1f38-ef66-4d82-a591-9fd1b6d72248 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:41.089576 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc1f38-ef66-4d82-a591-9fd1b6d72248 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:41.096026 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fc1f38-ef66-4d82-a591-9fd1b6d72248 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:41.097461 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fc1f38-ef66-4d82-a591-9fd1b6d72248 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:41.110115 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-91fc1f38-ef66-4d82-a591-9fd1b6d72248 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:28:41.110115 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:41.110115 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:41.110115 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:41.110115 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:41.110115 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:41.110115 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:41.110115 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:41.110115 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:41.110115 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:41.110115 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:41.110115 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:41.110115 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:28:41.110115 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:28:41.110115 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:28:41.110943 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:28:41.110943 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:41.110943 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:41.110943 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:41.110943 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:41.110943 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:41.110943 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:41.110943 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:41.110943 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:41.110943 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:41.110943 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:41.110943 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:41.110943 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:41.110943 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:41.110943 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:41.110943 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:41.110943 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:41.111789 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:28:41.111789 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:28:41.111789 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:28:41.111789 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:28:41.111789 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:28:41.111789 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:28:41.111789 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:28:41.111789 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:41.111789 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 260/514] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:28:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:41.112259 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc1f38-ef66-4d82-a591-9fd1b6d72248 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:41.162473 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:41.175086 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ffd19b-9ce4-4f4b-a8c4-eb16cbe5b27d None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:41.207461 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c2518d-59fe-45a4-88f7-2766ab0f172b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:41.208072 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c2518d-59fe-45a4-88f7-2766ab0f172b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:41.208284 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c2518d-59fe-45a4-88f7-2766ab0f172b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:41.208558 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c2518d-59fe-45a4-88f7-2766ab0f172b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:41.209763 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc1f38-ef66-4d82-a591-9fd1b6d72248 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:41.218786 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c2518d-59fe-45a4-88f7-2766ab0f172b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:41.220204 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c2518d-59fe-45a4-88f7-2766ab0f172b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:41.234017 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:41.234684 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-d7c2518d-59fe-45a4-88f7-2766ab0f172b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:41.234684 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:41.234684 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:41.234684 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:41.234684 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:41.234684 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:41.234684 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:41.234684 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:41.234684 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:41.234684 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:41.234684 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:41.234684 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:41.234684 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:28:41.234684 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:28:41.234684 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:28:41.234684 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:28:41.235571 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:41.235571 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:41.235571 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:41.235571 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:41.235571 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:41.235571 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:41.235571 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:41.235571 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:41.235571 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:41.235571 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:41.235571 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:41.235571 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:41.235571 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:41.235571 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:41.235571 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:41.235571 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:41.235571 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:28:41.236569 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:28:41.236569 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:41.236569 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:41.236569 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 255/515] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:41.236792 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c2518d-59fe-45a4-88f7-2766ab0f172b None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:41.246053 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e74823-d152-4738-a17a-bba918e12aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:41.246653 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e74823-d152-4738-a17a-bba918e12aa4 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:41.246930 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e74823-d152-4738-a17a-bba918e12aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:41.247201 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e74823-d152-4738-a17a-bba918e12aa4 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:41.247910 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e74823-d152-4738-a17a-bba918e12aa4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:41.248206 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e74823-d152-4738-a17a-bba918e12aa4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:41.249648 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e74823-d152-4738-a17a-bba918e12aa4 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:41.266842 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 261/516] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Sep 28 15:28:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:41.267624 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e74823-d152-4738-a17a-bba918e12aa4 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:41.338006 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c2518d-59fe-45a4-88f7-2766ab0f172b None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:41.370142 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e74823-d152-4738-a17a-bba918e12aa4 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:41.370733 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076b9ca8-5bb9-46a7-923d-6934678c00e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:41.371342 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b9ca8-5bb9-46a7-923d-6934678c00e2 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:41.371617 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b9ca8-5bb9-46a7-923d-6934678c00e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:41.371879 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b9ca8-5bb9-46a7-923d-6934678c00e2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:41.378991 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076b9ca8-5bb9-46a7-923d-6934678c00e2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:41.380389 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076b9ca8-5bb9-46a7-923d-6934678c00e2 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:41.383104 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:41.392652 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-076b9ca8-5bb9-46a7-923d-6934678c00e2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 28 15:28:41.392652 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:41.392652 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:41.392652 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:41.392652 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:41.392652 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:41.392652 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:41.392652 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:41.392652 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:41.392652 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:41.392652 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:41.392652 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:41.392652 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:28:41.392652 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:28:41.392652 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:28:41.392652 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:28:41.393544 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:41.393544 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:41.393544 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:28:41.393544 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:28:41.393544 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:28:41.393544 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:28:41.393544 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:41.393544 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:41.393544 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:41.393544 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:41.393544 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:41.393544 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:41.393544 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:41.393544 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:41.393544 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:41.393544 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:41.393544 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:41.393544 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:41.394459 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:41.394459 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:41.394459 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:28:41.394459 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:28:41.394459 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:28:41.394459 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:28:41.394459 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:28:41.394459 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:28:41.394459 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 28 15:28:41.394459 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:41.394459 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 256/517] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 15:28:41 2022] GET /identity/v3/users/neutron => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:41.395042 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076b9ca8-5bb9-46a7-923d-6934678c00e2 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:41.406772 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c89a88-73b3-46b1-a4f5-0e11d2073077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:41.407371 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c89a88-73b3-46b1-a4f5-0e11d2073077 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:41.407648 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c89a88-73b3-46b1-a4f5-0e11d2073077 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:41.408167 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c89a88-73b3-46b1-a4f5-0e11d2073077 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:41.408625 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c89a88-73b3-46b1-a4f5-0e11d2073077 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:41.408924 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c89a88-73b3-46b1-a4f5-0e11d2073077 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:41.410377 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c89a88-73b3-46b1-a4f5-0e11d2073077 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:41.426780 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 262/518] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Sep 28 15:28:41 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:41.427562 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c89a88-73b3-46b1-a4f5-0e11d2073077 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:41.470720 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:41.497175 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076b9ca8-5bb9-46a7-923d-6934678c00e2 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:41.526110 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c89a88-73b3-46b1-a4f5-0e11d2073077 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:41.530261 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd17901b-8abf-44e3-9244-4bb03e433b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:41.530852 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd17901b-8abf-44e3-9244-4bb03e433b3b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:41.531246 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd17901b-8abf-44e3-9244-4bb03e433b3b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:41.531517 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd17901b-8abf-44e3-9244-4bb03e433b3b None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:41.532196 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd17901b-8abf-44e3-9244-4bb03e433b3b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:41.532494 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd17901b-8abf-44e3-9244-4bb03e433b3b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:41.533930 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd17901b-8abf-44e3-9244-4bb03e433b3b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:41.551111 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 257/519] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 15:28:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:41.551735 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd17901b-8abf-44e3-9244-4bb03e433b3b None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:41.559264 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e318bfc6-6b60-4822-a72e-8091fe0c43b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:41.559864 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318bfc6-6b60-4822-a72e-8091fe0c43b6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:41.560139 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318bfc6-6b60-4822-a72e-8091fe0c43b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:41.560405 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318bfc6-6b60-4822-a72e-8091fe0c43b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:41.561028 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e318bfc6-6b60-4822-a72e-8091fe0c43b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:41.561323 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e318bfc6-6b60-4822-a72e-8091fe0c43b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:41.562696 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e318bfc6-6b60-4822-a72e-8091fe0c43b6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:41.574121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 263/520] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:28:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:41.574892 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e318bfc6-6b60-4822-a72e-8091fe0c43b6 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:41.612353 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:41.653202 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd17901b-8abf-44e3-9244-4bb03e433b3b None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:41.671879 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e318bfc6-6b60-4822-a72e-8091fe0c43b6 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:41.673336 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:41.687722 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a2d68e-7811-402d-a64f-e508bf24af3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:41.688324 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a2d68e-7811-402d-a64f-e508bf24af3d None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:41.688598 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a2d68e-7811-402d-a64f-e508bf24af3d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:41.688866 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a2d68e-7811-402d-a64f-e508bf24af3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:41.706325 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1571f2-bbac-44b0-ad86-91ee77f2f917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:41.706929 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1571f2-bbac-44b0-ad86-91ee77f2f917 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:41.707209 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1571f2-bbac-44b0-ad86-91ee77f2f917 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:41.707483 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1571f2-bbac-44b0-ad86-91ee77f2f917 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:41.708197 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1571f2-bbac-44b0-ad86-91ee77f2f917 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:41.708499 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1571f2-bbac-44b0-ad86-91ee77f2f917 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:41.709944 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1571f2-bbac-44b0-ad86-91ee77f2f917 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:41.726699 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 264/521] 10.222.0.52 () {56 vars in 1139 bytes} [Wed Sep 28 15:28:41 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:41.727989 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1571f2-bbac-44b0-ad86-91ee77f2f917 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:41.759445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a2d68e-7811-402d-a64f-e508bf24af3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:41.761003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a2d68e-7811-402d-a64f-e508bf24af3d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:41.818196 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:41.824022 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1571f2-bbac-44b0-ad86-91ee77f2f917 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:41.857514 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a90656-8b91-45aa-8e43-b4ed395ad710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:41.858132 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a90656-8b91-45aa-8e43-b4ed395ad710 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:41.858408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a90656-8b91-45aa-8e43-b4ed395ad710 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:41.858675 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a90656-8b91-45aa-8e43-b4ed395ad710 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:41.859739 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 258/522] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:28:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:28:41.860355 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a2d68e-7811-402d-a64f-e508bf24af3d None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:41.888719 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:41.927823 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a90656-8b91-45aa-8e43-b4ed395ad710 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:41.929408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a90656-8b91-45aa-8e43-b4ed395ad710 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:41.960265 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a2d68e-7811-402d-a64f-e508bf24af3d None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:41.994425 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fb7fd5-3433-4777-9d75-2e8e5f691914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:41.994948 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb7fd5-3433-4777-9d75-2e8e5f691914 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:41.995220 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb7fd5-3433-4777-9d75-2e8e5f691914 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:41.995485 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb7fd5-3433-4777-9d75-2e8e5f691914 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:41.996198 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fb7fd5-3433-4777-9d75-2e8e5f691914 None admin] RBAC: Adding query filter params (role.id=342b64fe0cdb4925b23c6cd461a152ac, scope.project.id=3dc6c5f5fcd34922b0d09cd2aaed6cf1, user.id=8daf14b64bf04b2ba74d2fe1fd41ff71) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:41.996504 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fb7fd5-3433-4777-9d75-2e8e5f691914 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:41.998007 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fb7fd5-3433-4777-9d75-2e8e5f691914 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:42.003173 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 259/523] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 15:28:41 2022] GET /identity/v3/role_assignments?user.id=8daf14b64bf04b2ba74d2fe1fd41ff71&scope.project.id=3dc6c5f5fcd34922b0d09cd2aaed6cf1&role.id=342b64fe0cdb4925b23c6cd461a152ac => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:42.003907 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fb7fd5-3433-4777-9d75-2e8e5f691914 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:42.032481 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 265/524] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:28:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:28:42.032481 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a90656-8b91-45aa-8e43-b4ed395ad710 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:42.045147 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:42.097992 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:42.133125 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fb7fd5-3433-4777-9d75-2e8e5f691914 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:42.144463 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a90656-8b91-45aa-8e43-b4ed395ad710 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:42.173481 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15961fc-9e90-49ef-bf16-ee40c35f988a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:42.174108 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15961fc-9e90-49ef-bf16-ee40c35f988a None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:42.174383 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15961fc-9e90-49ef-bf16-ee40c35f988a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:42.174649 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15961fc-9e90-49ef-bf16-ee40c35f988a None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:42.175287 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15961fc-9e90-49ef-bf16-ee40c35f988a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:42.176719 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15961fc-9e90-49ef-bf16-ee40c35f988a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:42.183133 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd7b1bd-fd97-48d1-be9d-98a3d2339924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:42.183754 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7b1bd-fd97-48d1-be9d-98a3d2339924 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:42.184035 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7b1bd-fd97-48d1-be9d-98a3d2339924 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:42.185019 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7b1bd-fd97-48d1-be9d-98a3d2339924 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:42.200525 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 260/525] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 15:28:42 2022] POST /identity/v3/users => generated 247 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:28:42.201731 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15961fc-9e90-49ef-bf16-ee40c35f988a None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:42.296497 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd7b1bd-fd97-48d1-be9d-98a3d2339924 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:42.298597 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd7b1bd-fd97-48d1-be9d-98a3d2339924 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:42.319334 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:42.344674 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15961fc-9e90-49ef-bf16-ee40c35f988a None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:42.377106 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:42.384683 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ade86a2-df97-4288-9cba-2b2dac97aba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:42.385392 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ade86a2-df97-4288-9cba-2b2dac97aba3 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:42.385732 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ade86a2-df97-4288-9cba-2b2dac97aba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:42.386113 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ade86a2-df97-4288-9cba-2b2dac97aba3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:42.398748 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ade86a2-df97-4288-9cba-2b2dac97aba3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:42.400348 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ade86a2-df97-4288-9cba-2b2dac97aba3 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:42.415929 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-1ade86a2-df97-4288-9cba-2b2dac97aba3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:42.415929 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:42.415929 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:42.415929 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:42.415929 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:42.415929 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:42.415929 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:42.415929 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:42.415929 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:42.415929 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:42.415929 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:42.415929 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:42.415929 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:28:42.415929 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:28:42.415929 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:28:42.415929 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:28:42.416861 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:42.416861 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:42.416861 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:42.416861 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:42.416861 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:42.416861 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:42.416861 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:42.416861 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:42.416861 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:42.416861 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:42.416861 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:42.416861 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:42.416861 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:42.416861 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:42.416861 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:42.416861 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:42.416861 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:28:42.417709 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:28:42.417709 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:42.417709 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:42.417709 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 261/526] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:42.418567 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ade86a2-df97-4288-9cba-2b2dac97aba3 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:42.438328 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 266/527] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:28:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:28:42.439257 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd7b1bd-fd97-48d1-be9d-98a3d2339924 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:42.536545 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ade86a2-df97-4288-9cba-2b2dac97aba3 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:42.569263 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd7b1bd-fd97-48d1-be9d-98a3d2339924 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:42.574084 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:42.580169 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e356dc-4661-4212-baf1-e5e0e7e701a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:42.580794 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e356dc-4661-4212-baf1-e5e0e7e701a8 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:42.581065 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e356dc-4661-4212-baf1-e5e0e7e701a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:42.581325 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e356dc-4661-4212-baf1-e5e0e7e701a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:42.595548 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e356dc-4661-4212-baf1-e5e0e7e701a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:42.597380 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e356dc-4661-4212-baf1-e5e0e7e701a8 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:42.616054 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-a2e356dc-4661-4212-baf1-e5e0e7e701a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:42.616054 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:42.616054 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:42.616054 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:42.616054 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:42.616054 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:42.616054 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:42.616054 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:42.616054 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:42.616054 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:42.616054 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:42.616054 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:42.616054 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:28:42.616054 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:28:42.616054 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:28:42.616054 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:28:42.617101 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:42.617101 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:42.617101 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:42.617101 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:42.617101 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:42.617101 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:42.617101 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:42.617101 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:42.617101 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:42.617101 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:42.617101 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:42.617101 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:42.617101 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:42.617101 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:42.617101 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:42.617101 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:42.617101 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:28:42.618935 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:28:42.618935 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:42.618935 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:42.620003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 262/528] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:42.624617 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e356dc-4661-4212-baf1-e5e0e7e701a8 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:42.627505 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae64cb5-86e8-435a-b178-4fcc723a3cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:42.629081 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae64cb5-86e8-435a-b178-4fcc723a3cd4 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:42.629369 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae64cb5-86e8-435a-b178-4fcc723a3cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:42.629637 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae64cb5-86e8-435a-b178-4fcc723a3cd4 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:42.630305 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae64cb5-86e8-435a-b178-4fcc723a3cd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:42.630603 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae64cb5-86e8-435a-b178-4fcc723a3cd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:42.632387 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae64cb5-86e8-435a-b178-4fcc723a3cd4 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:42.649928 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 267/529] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:28:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:42.651179 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae64cb5-86e8-435a-b178-4fcc723a3cd4 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:42.684095 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:42.752397 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e356dc-4661-4212-baf1-e5e0e7e701a8 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:42.776523 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae64cb5-86e8-435a-b178-4fcc723a3cd4 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:42.798337 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea2eee-e59e-445e-8bc3-91bae6e044ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:42.798969 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea2eee-e59e-445e-8bc3-91bae6e044ce None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:42.799244 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea2eee-e59e-445e-8bc3-91bae6e044ce None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:42.799517 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea2eee-e59e-445e-8bc3-91bae6e044ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:42.813668 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea2eee-e59e-445e-8bc3-91bae6e044ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:42.815719 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea2eee-e59e-445e-8bc3-91bae6e044ce None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:42.818053 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebafa71-4e86-41c9-8a4c-049fd6e34ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:42.818665 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebafa71-4e86-41c9-8a4c-049fd6e34ecb None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:42.818941 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebafa71-4e86-41c9-8a4c-049fd6e34ecb None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:42.819212 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebafa71-4e86-41c9-8a4c-049fd6e34ecb None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:42.820310 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebafa71-4e86-41c9-8a4c-049fd6e34ecb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:42.820556 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebafa71-4e86-41c9-8a4c-049fd6e34ecb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:42.822509 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebafa71-4e86-41c9-8a4c-049fd6e34ecb None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:42.833627 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-45ea2eee-e59e-445e-8bc3-91bae6e044ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:42.833627 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:42.833627 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:42.833627 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:42.833627 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:42.833627 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:42.833627 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:42.833627 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:42.833627 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:42.833627 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:42.833627 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:42.833627 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:42.833627 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:28:42.833627 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:28:42.833627 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:28:42.833627 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:28:42.834575 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:42.834575 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:42.834575 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:42.834575 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:42.834575 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:42.834575 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:42.834575 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:42.834575 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:42.834575 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:42.834575 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:42.834575 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:42.834575 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:42.834575 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:42.834575 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:42.834575 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:42.834575 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:42.834575 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:28:42.835428 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:28:42.835428 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:42.835428 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:42.836057 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 268/530] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:28:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:42.836904 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebafa71-4e86-41c9-8a4c-049fd6e34ecb None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:42.837935 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 263/531] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:42.846642 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea2eee-e59e-445e-8bc3-91bae6e044ce None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:42.884938 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:42.937735 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:42.962585 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebafa71-4e86-41c9-8a4c-049fd6e34ecb None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:42.963757 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea2eee-e59e-445e-8bc3-91bae6e044ce None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:42.999823 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7723c014-96be-4bb5-8785-f4d7671de964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:43.000714 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723c014-96be-4bb5-8785-f4d7671de964 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:43.001136 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723c014-96be-4bb5-8785-f4d7671de964 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:43.001646 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723c014-96be-4bb5-8785-f4d7671de964 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:43.009888 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7723c014-96be-4bb5-8785-f4d7671de964 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:43.011172 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b948d2-9a71-4af6-a442-92b0bee2e03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:43.011528 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7723c014-96be-4bb5-8785-f4d7671de964 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:43.012136 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b948d2-9a71-4af6-a442-92b0bee2e03f None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:43.012548 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b948d2-9a71-4af6-a442-92b0bee2e03f None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:43.012949 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b948d2-9a71-4af6-a442-92b0bee2e03f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:43.020888 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b948d2-9a71-4af6-a442-92b0bee2e03f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:43.022676 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b948d2-9a71-4af6-a442-92b0bee2e03f None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:43.023983 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-7723c014-96be-4bb5-8785-f4d7671de964 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:28:43.023983 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:43.023983 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:43.023983 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:43.023983 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:43.023983 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:43.023983 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:43.023983 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:43.023983 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:43.023983 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:43.023983 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:43.023983 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:43.023983 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:28:43.023983 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:28:43.023983 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:28:43.025483 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:28:43.025483 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:43.025483 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:43.025483 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:43.025483 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:43.025483 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:43.025483 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:43.025483 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:43.025483 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:43.025483 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:43.025483 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:43.025483 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:43.025483 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:43.025483 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:43.025483 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:43.025483 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:43.025483 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:43.026357 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:28:43.026357 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:28:43.026357 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:28:43.026357 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:28:43.026357 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:28:43.026357 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:28:43.026357 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:28:43.026357 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:43.030546 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 269/532] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:28:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:43.031522 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7723c014-96be-4bb5-8785-f4d7671de964 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:43.040892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-21b948d2-9a71-4af6-a442-92b0bee2e03f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:28:43.040892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:43.040892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:43.040892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:43.040892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:43.040892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:43.040892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:43.040892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:43.040892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:43.040892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:43.040892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:43.040892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:43.040892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:28:43.040892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:28:43.040892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:28:43.041948 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:28:43.041948 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:43.041948 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:43.041948 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:43.041948 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:43.041948 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:43.041948 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:43.041948 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:43.041948 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:43.041948 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:43.041948 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:43.041948 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:43.041948 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:43.041948 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:43.041948 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:43.041948 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:43.041948 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:43.042969 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:28:43.042969 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:28:43.042969 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:28:43.042969 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:28:43.042969 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:28:43.042969 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:28:43.042969 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:28:43.042969 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:43.045430 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 264/533] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:28:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:43.046811 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b948d2-9a71-4af6-a442-92b0bee2e03f None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:43.157536 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:43.163449 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7723c014-96be-4bb5-8785-f4d7671de964 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:43.164969 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b948d2-9a71-4af6-a442-92b0bee2e03f None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:43.205730 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87372741-1012-4200-834b-932cd92a97d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:43.206645 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87372741-1012-4200-834b-932cd92a97d2 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:43.207059 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87372741-1012-4200-834b-932cd92a97d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:43.207499 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87372741-1012-4200-834b-932cd92a97d2 None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:43.207638 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcbdc88-d119-45bb-bc8d-93b5c74cbf99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:43.208367 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcbdc88-d119-45bb-bc8d-93b5c74cbf99 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:43.208630 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87372741-1012-4200-834b-932cd92a97d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:43.208824 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcbdc88-d119-45bb-bc8d-93b5c74cbf99 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:43.209204 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcbdc88-d119-45bb-bc8d-93b5c74cbf99 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:43.209309 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87372741-1012-4200-834b-932cd92a97d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:43.210047 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcbdc88-d119-45bb-bc8d-93b5c74cbf99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:43.210494 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcbdc88-d119-45bb-bc8d-93b5c74cbf99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:43.211179 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87372741-1012-4200-834b-932cd92a97d2 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:43.212571 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcbdc88-d119-45bb-bc8d-93b5c74cbf99 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:43.222636 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:43.228378 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 265/534] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 15:28:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:43.229357 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87372741-1012-4200-834b-932cd92a97d2 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:43.230001 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 270/535] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:28:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:43.234359 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcbdc88-d119-45bb-bc8d-93b5c74cbf99 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:43.340133 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcbdc88-d119-45bb-bc8d-93b5c74cbf99 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:43.345133 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87372741-1012-4200-834b-932cd92a97d2 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:43.375446 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d0383-2611-41f8-9ba1-73d4415ee095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:43.376280 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d0383-2611-41f8-9ba1-73d4415ee095 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:43.376695 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d0383-2611-41f8-9ba1-73d4415ee095 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:43.377100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d0383-2611-41f8-9ba1-73d4415ee095 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:43.382394 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f163b45d-bd21-4f46-a75d-2b3d4cc26949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:43.382989 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163b45d-bd21-4f46-a75d-2b3d4cc26949 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:43.383263 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163b45d-bd21-4f46-a75d-2b3d4cc26949 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:43.383532 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163b45d-bd21-4f46-a75d-2b3d4cc26949 None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:43.384316 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f163b45d-bd21-4f46-a75d-2b3d4cc26949 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:43.384640 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f163b45d-bd21-4f46-a75d-2b3d4cc26949 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:43.385792 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88d0383-2611-41f8-9ba1-73d4415ee095 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:43.386051 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f163b45d-bd21-4f46-a75d-2b3d4cc26949 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:43.387515 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88d0383-2611-41f8-9ba1-73d4415ee095 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:43.398143 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 266/536] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 15:28:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:43.398890 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-e88d0383-2611-41f8-9ba1-73d4415ee095 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:28:43.398890 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:43.398890 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:43.398890 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:43.398890 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:43.398890 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:43.398890 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:43.398890 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:43.398890 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:43.398890 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:43.398890 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:43.398890 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:43.398890 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:28:43.398890 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:28:43.398890 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:28:43.399747 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:28:43.399747 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:43.399747 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:43.399747 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:43.399747 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:43.399747 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:43.399747 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:43.399747 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:43.399747 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:43.399747 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:43.399747 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:43.399747 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:43.399747 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:43.399747 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:43.399747 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:43.399747 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:43.399747 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:43.400576 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:28:43.400576 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:28:43.400576 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:28:43.400576 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:28:43.400576 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:28:43.400576 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:28:43.400576 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:28:43.400576 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:43.400965 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f163b45d-bd21-4f46-a75d-2b3d4cc26949 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:43.403112 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 271/537] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:28:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:43.415415 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d0383-2611-41f8-9ba1-73d4415ee095 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:43.418452 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:43.494565 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:43.513946 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f163b45d-bd21-4f46-a75d-2b3d4cc26949 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:43.546052 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d0383-2611-41f8-9ba1-73d4415ee095 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:43.552222 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ac37d8-b0c8-4190-8952-3f1623d6b0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:43.552985 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac37d8-b0c8-4190-8952-3f1623d6b0b8 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:43.553416 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac37d8-b0c8-4190-8952-3f1623d6b0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:43.553821 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac37d8-b0c8-4190-8952-3f1623d6b0b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:43.554684 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ac37d8-b0c8-4190-8952-3f1623d6b0b8 None admin] RBAC: Adding query filter params (role.id=342b64fe0cdb4925b23c6cd461a152ac, scope.project.id=3dc6c5f5fcd34922b0d09cd2aaed6cf1, user.id=c6a46bbd249c4434ad7d9442754f098a) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:43.554997 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ac37d8-b0c8-4190-8952-3f1623d6b0b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:43.557050 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ac37d8-b0c8-4190-8952-3f1623d6b0b8 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:43.569889 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 267/538] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 15:28:43 2022] GET /identity/v3/role_assignments?user.id=c6a46bbd249c4434ad7d9442754f098a&scope.project.id=3dc6c5f5fcd34922b0d09cd2aaed6cf1&role.id=342b64fe0cdb4925b23c6cd461a152ac => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:43.569889 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ac37d8-b0c8-4190-8952-3f1623d6b0b8 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:43.616680 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b22506-8182-4077-b8c9-9e10bb833c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:43.617297 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b22506-8182-4077-b8c9-9e10bb833c71 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:43.617572 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b22506-8182-4077-b8c9-9e10bb833c71 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:43.617836 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b22506-8182-4077-b8c9-9e10bb833c71 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:43.618727 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b22506-8182-4077-b8c9-9e10bb833c71 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:43.618997 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b22506-8182-4077-b8c9-9e10bb833c71 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:43.620483 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b22506-8182-4077-b8c9-9e10bb833c71 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:43.658483 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 272/539] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 15:28:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:43.670015 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b22506-8182-4077-b8c9-9e10bb833c71 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:43.726510 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:43.729675 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ac37d8-b0c8-4190-8952-3f1623d6b0b8 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:43.771468 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e68c4-289f-4db8-a6b9-92bbed4d580b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:43.772691 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e68c4-289f-4db8-a6b9-92bbed4d580b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:43.773197 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e68c4-289f-4db8-a6b9-92bbed4d580b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:43.773661 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e68c4-289f-4db8-a6b9-92bbed4d580b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:43.774655 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36e68c4-289f-4db8-a6b9-92bbed4d580b None admin] RBAC: Adding query filter params (role.id=342b64fe0cdb4925b23c6cd461a152ac, scope.project.id=3dc6c5f5fcd34922b0d09cd2aaed6cf1, user.id=01833709177e4e68a2fa52fb9963d3bf) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:43.775189 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36e68c4-289f-4db8-a6b9-92bbed4d580b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:43.776872 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36e68c4-289f-4db8-a6b9-92bbed4d580b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:43.788763 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 268/540] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 15:28:43 2022] GET /identity/v3/role_assignments?user.id=01833709177e4e68a2fa52fb9963d3bf&scope.project.id=3dc6c5f5fcd34922b0d09cd2aaed6cf1&role.id=342b64fe0cdb4925b23c6cd461a152ac => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:43.868591 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b22506-8182-4077-b8c9-9e10bb833c71 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:43.891146 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:43.913746 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802b6b65-e235-47d4-a9cf-bb9af5350c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:43.914395 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b6b65-e235-47d4-a9cf-bb9af5350c22 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:43.914671 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b6b65-e235-47d4-a9cf-bb9af5350c22 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:43.915729 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802b6b65-e235-47d4-a9cf-bb9af5350c22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:43.916297 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802b6b65-e235-47d4-a9cf-bb9af5350c22 None admin] RBAC: Adding query filter params (role.id=342b64fe0cdb4925b23c6cd461a152ac, scope.project.id=3dc6c5f5fcd34922b0d09cd2aaed6cf1, user.id=b13b0533444a449ead00a389c5f6bd1a) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:43.916601 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802b6b65-e235-47d4-a9cf-bb9af5350c22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:43.918057 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802b6b65-e235-47d4-a9cf-bb9af5350c22 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:43.924488 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 273/541] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 15:28:43 2022] GET /identity/v3/role_assignments?user.id=b13b0533444a449ead00a389c5f6bd1a&scope.project.id=3dc6c5f5fcd34922b0d09cd2aaed6cf1&role.id=342b64fe0cdb4925b23c6cd461a152ac => generated 259 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:44.350601 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887e16c-bb97-48d5-bd1e-41541afdc4d7 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:44.351126 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887e16c-bb97-48d5-bd1e-41541afdc4d7 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:44.351126 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887e16c-bb97-48d5-bd1e-41541afdc4d7 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:44.352678 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 269/542] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:44.367256 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddfe2a4-54ad-4416-9f05-947dda5609e9 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:44.367714 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddfe2a4-54ad-4416-9f05-947dda5609e9 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:44.367812 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddfe2a4-54ad-4416-9f05-947dda5609e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:44.402197 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-0ddfe2a4-54ad-4416-9f05-947dda5609e9 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:44.403390 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0ddfe2a4-54ad-4416-9f05-947dda5609e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:44.499030 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 274/543] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:28:44.606865 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f30ec8-b6d9-489f-8cd5-81a460f84f89 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:44.607209 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f30ec8-b6d9-489f-8cd5-81a460f84f89 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:44.607403 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f30ec8-b6d9-489f-8cd5-81a460f84f89 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:44.608974 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 270/544] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:44.614186 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf06de-2acd-4815-a60f-3667bfd31ff1 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:44.614471 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf06de-2acd-4815-a60f-3667bfd31ff1 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:44.614742 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf06de-2acd-4815-a60f-3667bfd31ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:44.646874 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-b6bf06de-2acd-4815-a60f-3667bfd31ff1 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:44.648118 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b6bf06de-2acd-4815-a60f-3667bfd31ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:44.735726 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 275/545] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:28:44.991649 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f30ec8-b6d9-489f-8cd5-81a460f84f89 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:45.118596 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f30ec8-b6d9-489f-8cd5-81a460f84f89 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:45.147231 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bf06de-2acd-4815-a60f-3667bfd31ff1 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:45.165874 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f76ad1-e4d4-4fa2-9936-6e1001c68dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:45.165874 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f76ad1-e4d4-4fa2-9936-6e1001c68dd1 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:45.165874 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f76ad1-e4d4-4fa2-9936-6e1001c68dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:45.165874 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f76ad1-e4d4-4fa2-9936-6e1001c68dd1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:45.185551 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f76ad1-e4d4-4fa2-9936-6e1001c68dd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:45.191962 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-40f76ad1-e4d4-4fa2-9936-6e1001c68dd1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:28:45.191962 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:45.191962 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:45.191962 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:45.191962 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:45.191962 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:45.191962 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:45.191962 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:45.191962 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:45.191962 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:45.191962 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:45.191962 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:45.191962 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:45.191962 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:45.191962 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:45.191962 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:45.192820 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:45.192820 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:45.192820 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:45.192820 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:45.192820 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:45.192820 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:45.192820 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:45.192820 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:45.192820 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:45.192820 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:45.192820 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:45.192820 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:45.192820 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:45.192820 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:45.192820 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:45.192820 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:45.193607 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:45.193607 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:45.193607 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:45.193607 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:45.193607 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:45.193607 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:45.193607 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:45.193607 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:45.193607 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:45.193607 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:45.193607 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:28:45.193607 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:45.198648 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 271/546] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 15:28:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:45.205240 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f76ad1-e4d4-4fa2-9936-6e1001c68dd1 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:45.212959 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:45.231964 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:45.331656 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bf06de-2acd-4815-a60f-3667bfd31ff1 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:45.386063 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5555794a-182d-46ac-927e-24be0b6b5cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:45.386945 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555794a-182d-46ac-927e-24be0b6b5cbe None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:45.386945 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555794a-182d-46ac-927e-24be0b6b5cbe None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:45.387446 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555794a-182d-46ac-927e-24be0b6b5cbe None admin] PATH_INFO: `/v3/roles/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:45.410771 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5555794a-182d-46ac-927e-24be0b6b5cbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:45.416853 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-5555794a-182d-46ac-927e-24be0b6b5cbe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:28:45.416853 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:45.416853 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:45.416853 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:45.416853 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:45.416853 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:45.416853 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:45.416853 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:45.416853 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:45.416853 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:45.416853 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:45.416853 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:45.416853 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:45.416853 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:45.416853 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:45.416853 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:45.418278 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:45.418278 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:45.418278 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:45.418278 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:45.418278 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:45.418278 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:45.418278 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:45.418278 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:45.418278 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:45.418278 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:45.418278 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:45.418278 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:45.418278 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:45.418278 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:45.418278 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:45.418278 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:45.419306 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:45.419306 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:45.419306 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:45.419306 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:45.419306 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:45.419306 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:45.419306 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:45.419306 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:45.419306 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:45.419306 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:45.419306 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:28:45.419306 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:45.419306 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 276/547] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 15:28:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:45.424832 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5555794a-182d-46ac-927e-24be0b6b5cbe None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:45.434360 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f76ad1-e4d4-4fa2-9936-6e1001c68dd1 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:45.503063 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5960f3fa-8713-4387-8d9f-ad4a64f1ce51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:45.503674 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5960f3fa-8713-4387-8d9f-ad4a64f1ce51 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:45.503950 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5960f3fa-8713-4387-8d9f-ad4a64f1ce51 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:45.504216 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5960f3fa-8713-4387-8d9f-ad4a64f1ce51 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:45.504868 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5960f3fa-8713-4387-8d9f-ad4a64f1ce51 None admin] RBAC: Adding query filter params (name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:45.505162 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5960f3fa-8713-4387-8d9f-ad4a64f1ce51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:45.506808 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5960f3fa-8713-4387-8d9f-ad4a64f1ce51 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:45.515422 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 272/548] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 15:28:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:45.522717 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5960f3fa-8713-4387-8d9f-ad4a64f1ce51 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:45.561964 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5555794a-182d-46ac-927e-24be0b6b5cbe None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:45.564995 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:45.608539 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:45.627064 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fffe8cf-15b4-4484-9203-5ab7e7a93833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:45.627678 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fffe8cf-15b4-4484-9203-5ab7e7a93833 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:45.627961 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fffe8cf-15b4-4484-9203-5ab7e7a93833 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:45.628236 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fffe8cf-15b4-4484-9203-5ab7e7a93833 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:45.628896 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fffe8cf-15b4-4484-9203-5ab7e7a93833 None admin] RBAC: Adding query filter params (name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:45.629198 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fffe8cf-15b4-4484-9203-5ab7e7a93833 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:45.630580 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fffe8cf-15b4-4484-9203-5ab7e7a93833 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:45.643916 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5960f3fa-8713-4387-8d9f-ad4a64f1ce51 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:45.645334 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 277/549] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 15:28:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:45.651337 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fffe8cf-15b4-4484-9203-5ab7e7a93833 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:45.683257 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c61ddbd-4f65-44fa-a237-36cabef173e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:45.683888 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c61ddbd-4f65-44fa-a237-36cabef173e3 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:45.684160 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c61ddbd-4f65-44fa-a237-36cabef173e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:45.684424 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c61ddbd-4f65-44fa-a237-36cabef173e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:45.695543 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c61ddbd-4f65-44fa-a237-36cabef173e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:45.697576 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c61ddbd-4f65-44fa-a237-36cabef173e3 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:45.714000 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-1c61ddbd-4f65-44fa-a237-36cabef173e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:45.714000 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:45.714000 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:45.714000 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:45.714000 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:45.714000 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:45.714000 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:45.714000 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:45.714000 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:45.714000 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:45.714000 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:45.714000 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:45.714000 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:28:45.714000 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:28:45.714000 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:28:45.714000 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:28:45.716343 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:45.716343 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:45.716343 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:45.716343 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:45.716343 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:45.716343 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:45.716343 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:45.716343 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:45.716343 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:45.716343 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:45.716343 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:45.716343 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:45.716343 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:45.716343 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:45.716343 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:45.716343 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:45.716343 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:28:45.718114 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:28:45.718114 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:45.718114 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:45.718114 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 273/550] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:45.722574 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c61ddbd-4f65-44fa-a237-36cabef173e3 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:45.838704 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fffe8cf-15b4-4484-9203-5ab7e7a93833 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:45.905821 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6658e16d-26e9-4c0c-8952-cafd351ce0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:45.906471 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6658e16d-26e9-4c0c-8952-cafd351ce0ac None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:45.906747 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6658e16d-26e9-4c0c-8952-cafd351ce0ac None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:45.907019 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6658e16d-26e9-4c0c-8952-cafd351ce0ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:45.919161 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c61ddbd-4f65-44fa-a237-36cabef173e3 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:45.929197 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6658e16d-26e9-4c0c-8952-cafd351ce0ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:45.930715 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6658e16d-26e9-4c0c-8952-cafd351ce0ac None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:45.947458 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-6658e16d-26e9-4c0c-8952-cafd351ce0ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:45.947458 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:45.947458 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:45.947458 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:45.947458 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:45.947458 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:45.947458 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:45.947458 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:45.947458 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:45.947458 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:45.947458 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:45.947458 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:45.947458 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:28:45.947458 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:28:45.947458 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:28:45.947458 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:28:45.948406 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:45.948406 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:45.948406 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:45.948406 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:45.948406 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:45.948406 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:45.948406 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:45.948406 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:45.948406 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:45.948406 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:45.948406 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:45.948406 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:45.948406 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:45.948406 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:45.948406 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:45.948406 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:45.948406 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:28:45.949425 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:28:45.949425 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:45.949425 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:45.949425 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 278/551] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:28:45.954757 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6658e16d-26e9-4c0c-8952-cafd351ce0ac None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:45.956334 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:45.972355 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36521b4b-cc5b-4b7d-b6a5-03964a4b4585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:45.973293 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36521b4b-cc5b-4b7d-b6a5-03964a4b4585 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:45.973797 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36521b4b-cc5b-4b7d-b6a5-03964a4b4585 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:45.974307 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36521b4b-cc5b-4b7d-b6a5-03964a4b4585 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:45.975491 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36521b4b-cc5b-4b7d-b6a5-03964a4b4585 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:45.975991 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36521b4b-cc5b-4b7d-b6a5-03964a4b4585 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:45.978129 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36521b4b-cc5b-4b7d-b6a5-03964a4b4585 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:45.994206 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 274/552] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:28:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:46.000112 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36521b4b-cc5b-4b7d-b6a5-03964a4b4585 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:46.001044 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:46.142442 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6658e16d-26e9-4c0c-8952-cafd351ce0ac None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:46.163993 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36521b4b-cc5b-4b7d-b6a5-03964a4b4585 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:46.179894 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70305933-e0fd-4402-b123-1876bbd54e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:46.180554 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70305933-e0fd-4402-b123-1876bbd54e57 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:46.180838 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70305933-e0fd-4402-b123-1876bbd54e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:46.181110 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70305933-e0fd-4402-b123-1876bbd54e57 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:46.181743 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70305933-e0fd-4402-b123-1876bbd54e57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:46.182071 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70305933-e0fd-4402-b123-1876bbd54e57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:46.183827 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70305933-e0fd-4402-b123-1876bbd54e57 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:46.202404 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 279/553] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:28:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:46.204064 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70305933-e0fd-4402-b123-1876bbd54e57 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:46.284172 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e68dc-1bb0-404b-80ce-6dd8f537bbd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:46.285173 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e68dc-1bb0-404b-80ce-6dd8f537bbd4 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:46.285633 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e68dc-1bb0-404b-80ce-6dd8f537bbd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:46.286125 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e68dc-1bb0-404b-80ce-6dd8f537bbd4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:46.299928 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:46.314724 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109e68dc-1bb0-404b-80ce-6dd8f537bbd4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:46.316321 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109e68dc-1bb0-404b-80ce-6dd8f537bbd4 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:46.326588 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70305933-e0fd-4402-b123-1876bbd54e57 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:46.351016 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-109e68dc-1bb0-404b-80ce-6dd8f537bbd4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 28 15:28:46.351016 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:46.351016 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:46.351016 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:46.351016 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:46.351016 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:46.351016 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:46.351016 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:46.351016 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:46.351016 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:46.351016 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:46.351016 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:46.351016 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:28:46.351016 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:28:46.351016 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:28:46.351016 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:28:46.351986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:46.351986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:46.351986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:28:46.351986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:28:46.351986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:28:46.351986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:28:46.351986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:46.351986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:46.351986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:46.351986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:46.351986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:46.351986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:46.351986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:46.351986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:46.351986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:46.351986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:46.351986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:46.351986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:46.353516 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:46.353516 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:46.353516 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:28:46.353516 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:28:46.353516 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:28:46.353516 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:28:46.353516 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:28:46.353516 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:28:46.353516 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 28 15:28:46.353516 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:46.353516 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 275/554] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Sep 28 15:28:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:46.358166 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e68dc-1bb0-404b-80ce-6dd8f537bbd4 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:46.368754 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7129247-148f-4300-830c-59ca42426efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:46.369629 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7129247-148f-4300-830c-59ca42426efa None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:46.376244 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7129247-148f-4300-830c-59ca42426efa None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:46.376244 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7129247-148f-4300-830c-59ca42426efa None admin] PATH_INFO: `/v3/users/swift` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:46.387891 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7129247-148f-4300-830c-59ca42426efa None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:46.389405 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7129247-148f-4300-830c-59ca42426efa None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:46.411443 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-f7129247-148f-4300-830c-59ca42426efa None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 28 15:28:46.411443 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:46.411443 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:46.411443 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:46.411443 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:46.411443 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:46.411443 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:46.411443 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:46.411443 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:46.411443 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:46.411443 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:46.411443 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:46.411443 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:28:46.411443 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:28:46.411443 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:28:46.411443 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:28:46.412596 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:46.412596 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:46.412596 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:28:46.412596 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:28:46.412596 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:28:46.412596 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:28:46.412596 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:46.412596 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:46.412596 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:46.412596 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:46.412596 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:46.412596 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:46.412596 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:46.412596 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:46.412596 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:46.412596 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:46.412596 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:46.412596 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:46.414174 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:46.414174 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:46.414174 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:28:46.414174 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:28:46.414174 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:28:46.414174 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:28:46.414174 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:28:46.414174 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:28:46.414174 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 28 15:28:46.414174 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:46.414174 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 280/555] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 15:28:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:28:46.415764 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1513f3e-eb37-44b5-8f8a-652b79346132 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:46.415948 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1513f3e-eb37-44b5-8f8a-652b79346132 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:46.416118 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1513f3e-eb37-44b5-8f8a-652b79346132 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:46.417650 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 281/556] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:46.423040 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d6b267-d0d1-4d72-96a5-0726173388af None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:46.423320 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d6b267-d0d1-4d72-96a5-0726173388af None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:46.423594 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d6b267-d0d1-4d72-96a5-0726173388af None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:46.424947 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 282/557] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:46.425679 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d6b267-d0d1-4d72-96a5-0726173388af None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:46.429135 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:46.495241 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e68dc-1bb0-404b-80ce-6dd8f537bbd4 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:46.531439 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0dc41f-bc96-43f6-82f7-461d81aea6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:46.532037 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0dc41f-bc96-43f6-82f7-461d81aea6c1 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:46.532312 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0dc41f-bc96-43f6-82f7-461d81aea6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:46.532576 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0dc41f-bc96-43f6-82f7-461d81aea6c1 None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:46.533279 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0dc41f-bc96-43f6-82f7-461d81aea6c1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:46.533575 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0dc41f-bc96-43f6-82f7-461d81aea6c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:46.535028 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0dc41f-bc96-43f6-82f7-461d81aea6c1 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:46.537792 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d6b267-d0d1-4d72-96a5-0726173388af None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:46.570272 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 276/558] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Sep 28 15:28:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:28:46.570431 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96564ac-5142-4114-a300-396af52c9a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:46.571122 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96564ac-5142-4114-a300-396af52c9a1d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:46.571407 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96564ac-5142-4114-a300-396af52c9a1d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:46.571749 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96564ac-5142-4114-a300-396af52c9a1d None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:46.572475 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96564ac-5142-4114-a300-396af52c9a1d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:46.572780 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96564ac-5142-4114-a300-396af52c9a1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:46.573245 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeebaf5-8838-4e41-a2fd-bf75e1395b16 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:46.573531 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeebaf5-8838-4e41-a2fd-bf75e1395b16 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:46.573804 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeebaf5-8838-4e41-a2fd-bf75e1395b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:46.574227 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96564ac-5142-4114-a300-396af52c9a1d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:46.582775 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:46.591709 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-5aeebaf5-8838-4e41-a2fd-bf75e1395b16 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:46.592650 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 283/559] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Sep 28 15:28:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:46.592906 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-5aeebaf5-8838-4e41-a2fd-bf75e1395b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:46.593833 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc5865-e8a5-4128-8632-3cee6d377459 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:46.594153 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc5865-e8a5-4128-8632-3cee6d377459 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:46.594421 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc5865-e8a5-4128-8632-3cee6d377459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:46.611882 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-77cc5865-e8a5-4128-8632-3cee6d377459 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:46.613040 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-77cc5865-e8a5-4128-8632-3cee6d377459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:46.659482 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 277/560] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:46.660800 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e68de-843f-48d3-9773-69f3616ac2d9 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:46.661146 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e68de-843f-48d3-9773-69f3616ac2d9 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:46.661539 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e68de-843f-48d3-9773-69f3616ac2d9 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:46.662888 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 278/561] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:46.663413 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e68de-843f-48d3-9773-69f3616ac2d9 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:46.679753 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 284/562] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:46.680489 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc5865-e8a5-4128-8632-3cee6d377459 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:46.704667 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:46.799513 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e68de-843f-48d3-9773-69f3616ac2d9 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:46.808314 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc5865-e8a5-4128-8632-3cee6d377459 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:46.835153 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b9a445-258c-467d-babf-bec2e703dbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:46.835851 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9a445-258c-467d-babf-bec2e703dbae None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:46.836189 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9a445-258c-467d-babf-bec2e703dbae None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:46.836521 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9a445-258c-467d-babf-bec2e703dbae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:46.844108 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:46.847959 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b9a445-258c-467d-babf-bec2e703dbae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:46.849533 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b9a445-258c-467d-babf-bec2e703dbae None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:46.855593 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873ee967-f007-4a66-8677-1dcb814a8487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:46.856429 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ee967-f007-4a66-8677-1dcb814a8487 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:46.856897 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ee967-f007-4a66-8677-1dcb814a8487 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:46.857391 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ee967-f007-4a66-8677-1dcb814a8487 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:46.866132 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-72b9a445-258c-467d-babf-bec2e703dbae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:46.866132 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:46.866132 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:46.866132 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:46.866132 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:46.866132 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:46.866132 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:46.866132 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:46.866132 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:46.866132 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:46.866132 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:46.866132 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:46.866132 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:28:46.866132 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:28:46.866132 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:28:46.866132 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:28:46.867191 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:46.867191 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:46.867191 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:46.867191 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:46.867191 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:46.867191 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:46.867191 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:46.867191 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:46.867191 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:46.867191 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:46.867191 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:46.867191 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:46.867191 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:46.867191 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:46.867191 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:46.867191 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:46.867191 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:28:46.868950 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:28:46.868950 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:46.868950 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:46.868950 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 279/563] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:46.869818 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb85df7-a7d1-4f94-915a-056411ed28a6 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:46.870193 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb85df7-a7d1-4f94-915a-056411ed28a6 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:46.870552 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb85df7-a7d1-4f94-915a-056411ed28a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:46.890477 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-feb85df7-a7d1-4f94-915a-056411ed28a6 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:46.891988 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-feb85df7-a7d1-4f94-915a-056411ed28a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:46.948944 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873ee967-f007-4a66-8677-1dcb814a8487 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:46.950431 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873ee967-f007-4a66-8677-1dcb814a8487 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:46.964548 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 280/564] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:46.965229 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb85df7-a7d1-4f94-915a-056411ed28a6 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:46.975335 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:47.061978 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 285/565] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:28:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:28:47.062862 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873ee967-f007-4a66-8677-1dcb814a8487 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:47.082010 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb85df7-a7d1-4f94-915a-056411ed28a6 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:47.094329 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:47.116344 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c99452-dc08-480d-8d86-0b0e2b4080d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:47.116935 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c99452-dc08-480d-8d86-0b0e2b4080d3 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:47.117205 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c99452-dc08-480d-8d86-0b0e2b4080d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:47.117472 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c99452-dc08-480d-8d86-0b0e2b4080d3 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:47.118211 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c99452-dc08-480d-8d86-0b0e2b4080d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:47.118447 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c99452-dc08-480d-8d86-0b0e2b4080d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:47.119802 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c99452-dc08-480d-8d86-0b0e2b4080d3 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:47.136039 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 281/566] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:28:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:47.136039 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c99452-dc08-480d-8d86-0b0e2b4080d3 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:47.179543 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873ee967-f007-4a66-8677-1dcb814a8487 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:47.209388 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:47.216568 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e1c84d-a7db-48b4-9215-50a39669145b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:47.217201 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e1c84d-a7db-48b4-9215-50a39669145b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:47.217482 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e1c84d-a7db-48b4-9215-50a39669145b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:47.217752 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e1c84d-a7db-48b4-9215-50a39669145b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:47.227932 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e1c84d-a7db-48b4-9215-50a39669145b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:47.232902 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-56e1c84d-a7db-48b4-9215-50a39669145b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:28:47.232902 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:47.232902 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:47.232902 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:47.232902 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:47.232902 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:47.232902 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:47.232902 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:47.232902 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:47.232902 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:47.232902 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:47.232902 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:47.232902 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:47.232902 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:47.232902 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:47.232902 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:47.234055 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:47.234055 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:47.234055 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:47.234055 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:47.234055 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:47.234055 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:47.234055 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:47.234055 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:47.234055 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:47.234055 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:47.234055 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:47.234055 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:47.234055 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:47.234055 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:47.234055 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:47.234055 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:47.234891 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:47.234891 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:47.234891 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:47.234891 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:47.234891 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:47.234891 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:47.234891 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:47.234891 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:47.234891 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:47.234891 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:47.234891 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:28:47.234891 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:47.234891 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 286/567] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 15:28:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:47.235559 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e1c84d-a7db-48b4-9215-50a39669145b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:47.249661 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c99452-dc08-480d-8d86-0b0e2b4080d3 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:47.283106 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d1e69d-08e3-4915-8053-48df68e9e2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:47.283730 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1e69d-08e3-4915-8053-48df68e9e2ef None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:47.284006 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1e69d-08e3-4915-8053-48df68e9e2ef None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:47.284271 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1e69d-08e3-4915-8053-48df68e9e2ef None admin] PATH_INFO: `/v3/roles/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:47.294712 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d1e69d-08e3-4915-8053-48df68e9e2ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:47.299854 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-c9d1e69d-08e3-4915-8053-48df68e9e2ef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:28:47.299854 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:47.299854 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:47.299854 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:47.299854 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:47.299854 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:47.299854 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:47.299854 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:47.299854 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:47.299854 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:47.299854 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:47.299854 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:47.299854 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:47.299854 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:47.299854 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:47.299854 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:47.301020 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:47.301020 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:47.301020 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:47.301020 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:47.301020 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:47.301020 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:47.301020 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:47.301020 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:47.301020 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:47.301020 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:47.301020 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:47.301020 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:47.301020 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:47.301020 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:47.301020 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:47.301020 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:47.302187 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:47.302187 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:47.302187 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:47.302187 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:47.302187 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:47.302187 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:47.302187 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:47.302187 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:47.302187 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:47.302187 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:47.302187 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:28:47.302187 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:47.302187 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 282/568] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 15:28:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:47.302187 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d1e69d-08e3-4915-8053-48df68e9e2ef None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:47.324223 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:47.338968 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e1c84d-a7db-48b4-9215-50a39669145b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:47.371406 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728de22c-2c78-4e12-9185-b9e74962bfcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:47.372017 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728de22c-2c78-4e12-9185-b9e74962bfcb None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:47.372293 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728de22c-2c78-4e12-9185-b9e74962bfcb None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:47.372559 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728de22c-2c78-4e12-9185-b9e74962bfcb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:47.382380 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728de22c-2c78-4e12-9185-b9e74962bfcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:47.384364 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728de22c-2c78-4e12-9185-b9e74962bfcb None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:47.398511 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-728de22c-2c78-4e12-9185-b9e74962bfcb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:47.398511 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:47.398511 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:47.398511 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:47.398511 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:47.398511 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:47.398511 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:47.398511 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:47.398511 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:47.398511 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:47.398511 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:47.398511 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:47.398511 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:28:47.398511 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:28:47.398511 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:28:47.398511 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:28:47.400011 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:47.400011 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:47.400011 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:47.400011 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:47.400011 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:47.400011 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:47.400011 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:47.400011 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:47.400011 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:47.400011 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:47.400011 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:47.400011 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:47.400011 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:47.400011 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:47.400011 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:47.400011 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:47.400011 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:28:47.400887 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:28:47.400887 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:47.400887 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:47.401055 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d1e69d-08e3-4915-8053-48df68e9e2ef None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:47.404356 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 287/569] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:47.405186 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728de22c-2c78-4e12-9185-b9e74962bfcb None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:47.425588 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:47.433971 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eb3a95-5bf3-4c50-a5c7-cfaf13f46a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:47.434645 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb3a95-5bf3-4c50-a5c7-cfaf13f46a95 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:47.434921 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb3a95-5bf3-4c50-a5c7-cfaf13f46a95 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:47.435219 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb3a95-5bf3-4c50-a5c7-cfaf13f46a95 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:47.441642 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eb3a95-5bf3-4c50-a5c7-cfaf13f46a95 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:47.443121 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eb3a95-5bf3-4c50-a5c7-cfaf13f46a95 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:47.454618 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-53eb3a95-5bf3-4c50-a5c7-cfaf13f46a95 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:28:47.454618 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:47.454618 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:47.454618 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:47.454618 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:47.454618 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:47.454618 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:47.454618 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:47.454618 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:47.454618 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:47.454618 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:47.454618 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:47.454618 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:28:47.454618 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:28:47.454618 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:28:47.455492 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:28:47.455492 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:47.455492 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:47.455492 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:47.455492 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:47.455492 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:47.455492 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:47.455492 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:47.455492 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:47.455492 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:47.455492 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:47.455492 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:47.455492 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:47.455492 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:47.455492 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:47.455492 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:47.455492 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:47.456357 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:28:47.456357 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:28:47.456357 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:28:47.456357 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:28:47.456357 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:28:47.456357 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:28:47.456357 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:28:47.456357 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:47.456357 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 283/570] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:28:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:47.456824 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eb3a95-5bf3-4c50-a5c7-cfaf13f46a95 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:47.506866 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728de22c-2c78-4e12-9185-b9e74962bfcb None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:47.540335 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f45c0-3e5b-4fef-86c3-9d905a705cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:47.540956 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f45c0-3e5b-4fef-86c3-9d905a705cb8 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:47.541234 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f45c0-3e5b-4fef-86c3-9d905a705cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:47.541508 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f45c0-3e5b-4fef-86c3-9d905a705cb8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:47.549008 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:47.553360 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9f45c0-3e5b-4fef-86c3-9d905a705cb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:47.558381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-fe9f45c0-3e5b-4fef-86c3-9d905a705cb8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:28:47.558381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:47.558381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:47.558381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:47.558381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:47.558381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:47.558381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:47.558381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:47.558381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:47.558381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:47.558381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:47.558381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:47.558381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:47.558381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:47.558381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:47.558381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:47.559477 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:47.559477 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:47.559477 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:47.559477 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:47.559477 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:47.559477 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:47.559477 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:47.559477 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:47.559477 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:47.559477 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:47.559477 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:47.559477 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:47.559477 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:47.559477 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:47.559477 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:47.559477 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:47.560315 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:47.560315 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:47.560315 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:47.560315 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:47.560315 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:47.560315 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:47.560315 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:47.560315 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:47.560315 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:47.560315 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:47.560315 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:28:47.560315 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:47.560315 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 288/571] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 15:28:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:47.560991 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eb3a95-5bf3-4c50-a5c7-cfaf13f46a95 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:47.561402 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f45c0-3e5b-4fef-86c3-9d905a705cb8 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:47.593087 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b10ad1-88a2-4c72-bda8-3771c58c17a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:47.593683 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b10ad1-88a2-4c72-bda8-3771c58c17a0 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:47.593989 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b10ad1-88a2-4c72-bda8-3771c58c17a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:47.594260 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b10ad1-88a2-4c72-bda8-3771c58c17a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:47.594914 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b10ad1-88a2-4c72-bda8-3771c58c17a0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:47.595215 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b10ad1-88a2-4c72-bda8-3771c58c17a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:47.596573 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b10ad1-88a2-4c72-bda8-3771c58c17a0 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:47.605098 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 284/572] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 15:28:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:47.606072 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b10ad1-88a2-4c72-bda8-3771c58c17a0 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:47.658160 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:47.666790 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f45c0-3e5b-4fef-86c3-9d905a705cb8 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:47.700013 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74449d93-2659-47b0-bf2e-93e67a05fcb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:47.700616 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74449d93-2659-47b0-bf2e-93e67a05fcb7 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:47.700952 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74449d93-2659-47b0-bf2e-93e67a05fcb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:47.701243 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74449d93-2659-47b0-bf2e-93e67a05fcb7 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:47.701918 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74449d93-2659-47b0-bf2e-93e67a05fcb7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:47.702227 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74449d93-2659-47b0-bf2e-93e67a05fcb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:47.703608 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74449d93-2659-47b0-bf2e-93e67a05fcb7 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:47.709471 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b10ad1-88a2-4c72-bda8-3771c58c17a0 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:47.711251 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 289/573] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 15:28:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:47.712576 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74449d93-2659-47b0-bf2e-93e67a05fcb7 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:47.744280 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b034e0c-fd2f-4a5b-876c-c42600707a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:47.744877 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b034e0c-fd2f-4a5b-876c-c42600707a3d None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:47.745155 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b034e0c-fd2f-4a5b-876c-c42600707a3d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:47.745425 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b034e0c-fd2f-4a5b-876c-c42600707a3d None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:47.746089 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b034e0c-fd2f-4a5b-876c-c42600707a3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:47.746390 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b034e0c-fd2f-4a5b-876c-c42600707a3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:47.747764 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b034e0c-fd2f-4a5b-876c-c42600707a3d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:47.759921 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 285/574] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:28:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:47.760634 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b034e0c-fd2f-4a5b-876c-c42600707a3d None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:47.770935 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:47.812570 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74449d93-2659-47b0-bf2e-93e67a05fcb7 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:47.844673 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90577adc-548b-4dd1-adf1-e8684976e8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:47.845291 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90577adc-548b-4dd1-adf1-e8684976e8c5 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:47.845577 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90577adc-548b-4dd1-adf1-e8684976e8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:47.845865 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90577adc-548b-4dd1-adf1-e8684976e8c5 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:47.846556 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90577adc-548b-4dd1-adf1-e8684976e8c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:47.846859 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90577adc-548b-4dd1-adf1-e8684976e8c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:47.848286 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90577adc-548b-4dd1-adf1-e8684976e8c5 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:47.856914 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b034e0c-fd2f-4a5b-876c-c42600707a3d None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:47.861623 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 290/575] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 15:28:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:47.861623 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90577adc-548b-4dd1-adf1-e8684976e8c5 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:47.869589 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:47.892499 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34fadb7-8d5a-438b-84d2-a9d40369c567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:47.893092 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34fadb7-8d5a-438b-84d2-a9d40369c567 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:47.893362 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34fadb7-8d5a-438b-84d2-a9d40369c567 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:47.893628 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34fadb7-8d5a-438b-84d2-a9d40369c567 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:47.894303 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34fadb7-8d5a-438b-84d2-a9d40369c567 None admin] RBAC: Adding query filter params (name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:47.894598 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34fadb7-8d5a-438b-84d2-a9d40369c567 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:47.895933 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34fadb7-8d5a-438b-84d2-a9d40369c567 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:47.903504 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 286/576] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 15:28:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:47.904268 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34fadb7-8d5a-438b-84d2-a9d40369c567 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:47.964928 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90577adc-548b-4dd1-adf1-e8684976e8c5 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:47.985715 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:48.000377 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac7b7ab-6f84-4608-bbbf-3d03b1e0951b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:48.001018 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac7b7ab-6f84-4608-bbbf-3d03b1e0951b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:48.001289 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac7b7ab-6f84-4608-bbbf-3d03b1e0951b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:48.001553 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac7b7ab-6f84-4608-bbbf-3d03b1e0951b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:48.003401 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34fadb7-8d5a-438b-84d2-a9d40369c567 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:48.011509 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac7b7ab-6f84-4608-bbbf-3d03b1e0951b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:48.012872 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac7b7ab-6f84-4608-bbbf-3d03b1e0951b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:48.027224 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-2ac7b7ab-6f84-4608-bbbf-3d03b1e0951b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:48.027224 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:48.027224 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:48.027224 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:48.027224 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:48.027224 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:48.027224 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:48.027224 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:48.027224 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:48.027224 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:48.027224 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:48.027224 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:48.027224 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:28:48.027224 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:28:48.027224 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:28:48.027224 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:28:48.028326 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:48.028326 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:48.028326 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:48.028326 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:48.028326 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:48.028326 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:48.028326 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:48.028326 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:48.028326 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:48.028326 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:48.028326 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:48.028326 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:48.028326 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:48.028326 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:48.028326 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:48.028326 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:48.028326 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:28:48.029287 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:28:48.029287 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:48.029287 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:48.030119 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 291/577] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:48.030927 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac7b7ab-6f84-4608-bbbf-3d03b1e0951b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:48.037028 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c25a14-beee-40e5-b488-7eb5046d7981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:48.037651 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c25a14-beee-40e5-b488-7eb5046d7981 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:48.037964 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c25a14-beee-40e5-b488-7eb5046d7981 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:48.038235 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c25a14-beee-40e5-b488-7eb5046d7981 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:48.049072 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c25a14-beee-40e5-b488-7eb5046d7981 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:48.050529 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c25a14-beee-40e5-b488-7eb5046d7981 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:48.064251 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-f9c25a14-beee-40e5-b488-7eb5046d7981 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:48.064251 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:48.064251 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:48.064251 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:48.064251 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:48.064251 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:48.064251 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:48.064251 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:48.064251 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:48.064251 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:48.064251 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:48.064251 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:48.064251 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:28:48.064251 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:28:48.064251 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:28:48.064251 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:28:48.065140 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:48.065140 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:48.065140 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:48.065140 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:48.065140 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:48.065140 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:48.065140 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:48.065140 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:48.065140 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:48.065140 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:48.065140 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:48.065140 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:48.065140 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:48.065140 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:48.065140 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:48.065140 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:48.065140 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:28:48.066045 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:28:48.066045 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:48.066045 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:48.066045 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 287/578] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:48.066658 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c25a14-beee-40e5-b488-7eb5046d7981 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:48.094193 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:48.128311 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac7b7ab-6f84-4608-bbbf-3d03b1e0951b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:48.163229 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f726f-45fa-4bd7-b914-957914a9d1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:48.163857 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f726f-45fa-4bd7-b914-957914a9d1bc None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:48.164135 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f726f-45fa-4bd7-b914-957914a9d1bc None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:48.164403 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f726f-45fa-4bd7-b914-957914a9d1bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:48.168689 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c25a14-beee-40e5-b488-7eb5046d7981 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:48.171404 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6f726f-45fa-4bd7-b914-957914a9d1bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:48.172834 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6f726f-45fa-4bd7-b914-957914a9d1bc None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:48.183695 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-3d6f726f-45fa-4bd7-b914-957914a9d1bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:28:48.183695 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:48.183695 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:48.183695 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:48.183695 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:48.183695 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:48.183695 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:48.183695 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:48.183695 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:48.183695 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:48.183695 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:48.183695 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:48.183695 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:28:48.183695 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:28:48.183695 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:28:48.184667 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:28:48.184667 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:48.184667 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:48.184667 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:48.184667 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:48.184667 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:48.184667 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:48.184667 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:48.184667 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:48.184667 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:48.184667 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:48.184667 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:48.184667 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:48.184667 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:48.184667 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:48.184667 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:48.184667 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:48.185554 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:28:48.185554 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:28:48.185554 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:28:48.185554 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:28:48.185554 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:28:48.185554 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:28:48.185554 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:28:48.185554 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:48.185554 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 292/579] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:28:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:48.185554 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f726f-45fa-4bd7-b914-957914a9d1bc None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:48.203819 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf014290-f5b4-4081-9c4e-80647224aaa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:48.204471 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf014290-f5b4-4081-9c4e-80647224aaa1 None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:48.204747 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf014290-f5b4-4081-9c4e-80647224aaa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:48.205020 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf014290-f5b4-4081-9c4e-80647224aaa1 None admin] PATH_INFO: `/v3/projects/3dc6c5f5fcd34922b0d09cd2aaed6cf1/users/8daf14b64bf04b2ba74d2fe1fd41ff71/roles/342b64fe0cdb4925b23c6cd461a152ac` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:48.215940 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:48.219759 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf014290-f5b4-4081-9c4e-80647224aaa1 None admin] RBAC: Authorizing `identity:create_grant(project_id=3dc6c5f5fcd34922b0d09cd2aaed6cf1, user_id=8daf14b64bf04b2ba74d2fe1fd41ff71, role_id=342b64fe0cdb4925b23c6cd461a152ac)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:48.221203 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf014290-f5b4-4081-9c4e-80647224aaa1 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:48.234128 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 288/580] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 15:28:48 2022] PUT /identity/v3/projects/3dc6c5f5fcd34922b0d09cd2aaed6cf1/users/8daf14b64bf04b2ba74d2fe1fd41ff71/roles/342b64fe0cdb4925b23c6cd461a152ac => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:28:48.234882 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf014290-f5b4-4081-9c4e-80647224aaa1 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:48.300725 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f726f-45fa-4bd7-b914-957914a9d1bc None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:48.336001 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:48.347525 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7479a414-6dfa-4e51-8a83-da895e89ddfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:48.348201 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479a414-6dfa-4e51-8a83-da895e89ddfc None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:48.348490 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479a414-6dfa-4e51-8a83-da895e89ddfc None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:48.348764 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479a414-6dfa-4e51-8a83-da895e89ddfc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:48.360707 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7479a414-6dfa-4e51-8a83-da895e89ddfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:48.362303 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7479a414-6dfa-4e51-8a83-da895e89ddfc None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:48.369392 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf014290-f5b4-4081-9c4e-80647224aaa1 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:48.379485 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-7479a414-6dfa-4e51-8a83-da895e89ddfc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:48.379485 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:48.379485 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:48.379485 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:48.379485 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:48.379485 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:48.379485 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:48.379485 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:48.379485 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:48.379485 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:48.379485 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:48.379485 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:48.379485 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:28:48.379485 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:28:48.379485 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:28:48.379485 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:28:48.380545 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:48.380545 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:48.380545 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:48.380545 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:48.380545 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:48.380545 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:48.380545 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:48.380545 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:48.380545 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:48.380545 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:48.380545 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:48.380545 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:48.380545 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:48.380545 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:48.380545 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:48.380545 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:48.380545 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:28:48.381686 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:28:48.381686 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:48.381686 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:48.386313 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 293/581] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:48.388169 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7479a414-6dfa-4e51-8a83-da895e89ddfc None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:48.415752 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038d0fb0-b4e6-49fc-acf6-4ebe46473b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:48.416482 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038d0fb0-b4e6-49fc-acf6-4ebe46473b6d None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:48.417107 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038d0fb0-b4e6-49fc-acf6-4ebe46473b6d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:48.417107 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038d0fb0-b4e6-49fc-acf6-4ebe46473b6d None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:48.417752 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038d0fb0-b4e6-49fc-acf6-4ebe46473b6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:48.418058 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038d0fb0-b4e6-49fc-acf6-4ebe46473b6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:48.419552 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038d0fb0-b4e6-49fc-acf6-4ebe46473b6d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:48.433074 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 289/582] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:28:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:48.434063 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038d0fb0-b4e6-49fc-acf6-4ebe46473b6d None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:48.483962 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:48.502355 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7479a414-6dfa-4e51-8a83-da895e89ddfc None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:48.537143 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94770a2f-dfe2-4349-ad5f-1fc0988ec015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:48.537744 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94770a2f-dfe2-4349-ad5f-1fc0988ec015 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:48.538050 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94770a2f-dfe2-4349-ad5f-1fc0988ec015 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:48.538318 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94770a2f-dfe2-4349-ad5f-1fc0988ec015 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:48.538698 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038d0fb0-b4e6-49fc-acf6-4ebe46473b6d None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:48.539008 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94770a2f-dfe2-4349-ad5f-1fc0988ec015 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:48.539308 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94770a2f-dfe2-4349-ad5f-1fc0988ec015 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:48.541569 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94770a2f-dfe2-4349-ad5f-1fc0988ec015 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:48.553755 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 294/583] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:28:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:48.554717 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94770a2f-dfe2-4349-ad5f-1fc0988ec015 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:48.582381 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3c338e-42a7-42b9-a2da-8e2ba9d40b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:48.583653 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c338e-42a7-42b9-a2da-8e2ba9d40b3c None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:48.584042 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c338e-42a7-42b9-a2da-8e2ba9d40b3c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:48.584383 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c338e-42a7-42b9-a2da-8e2ba9d40b3c None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:48.585712 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3c338e-42a7-42b9-a2da-8e2ba9d40b3c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:48.586037 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3c338e-42a7-42b9-a2da-8e2ba9d40b3c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:48.587694 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3c338e-42a7-42b9-a2da-8e2ba9d40b3c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:48.589562 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:48.603004 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 290/584] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 15:28:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:48.608565 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3c338e-42a7-42b9-a2da-8e2ba9d40b3c None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:48.662923 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94770a2f-dfe2-4349-ad5f-1fc0988ec015 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:48.700761 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a691a4-4dea-4f5c-abc7-216a00929938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:48.701365 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a691a4-4dea-4f5c-abc7-216a00929938 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:48.701650 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a691a4-4dea-4f5c-abc7-216a00929938 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:48.701952 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a691a4-4dea-4f5c-abc7-216a00929938 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:48.702590 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a691a4-4dea-4f5c-abc7-216a00929938 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:48.703149 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a691a4-4dea-4f5c-abc7-216a00929938 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:48.704355 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a691a4-4dea-4f5c-abc7-216a00929938 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:48.717137 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 295/585] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:28:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:48.717829 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a691a4-4dea-4f5c-abc7-216a00929938 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:48.723457 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:48.723709 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3c338e-42a7-42b9-a2da-8e2ba9d40b3c None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:48.760596 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f400a90-95ad-4019-86b3-9955a5ce88d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:48.761225 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f400a90-95ad-4019-86b3-9955a5ce88d0 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:48.761495 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f400a90-95ad-4019-86b3-9955a5ce88d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:48.761759 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f400a90-95ad-4019-86b3-9955a5ce88d0 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:48.770137 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f400a90-95ad-4019-86b3-9955a5ce88d0 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:48.771676 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f400a90-95ad-4019-86b3-9955a5ce88d0 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:48.784072 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-2f400a90-95ad-4019-86b3-9955a5ce88d0 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 28 15:28:48.784072 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:48.784072 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:48.784072 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:48.784072 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:48.784072 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:48.784072 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:48.784072 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:48.784072 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:48.784072 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:48.784072 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:48.784072 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:48.784072 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:28:48.784072 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:28:48.784072 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:28:48.784072 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:28:48.785205 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:48.785205 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:48.785205 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:28:48.785205 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:28:48.785205 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:28:48.785205 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:28:48.785205 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:48.785205 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:48.785205 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:48.785205 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:48.785205 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:48.785205 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:48.785205 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:48.785205 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:48.785205 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:48.785205 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:48.785205 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:48.785205 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:48.786334 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:48.786334 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:48.786334 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:28:48.786334 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:28:48.786334 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:28:48.786334 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:28:48.786334 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:28:48.786334 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:28:48.786334 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 28 15:28:48.786334 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:48.786334 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 291/586] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 15:28:48 2022] GET /identity/v3/users/neutron => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:48.786334 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f400a90-95ad-4019-86b3-9955a5ce88d0 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:48.827362 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a691a4-4dea-4f5c-abc7-216a00929938 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:48.843840 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:48.862132 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a335430c-422d-4c8b-8a4f-0264c8c19f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:48.862763 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a335430c-422d-4c8b-8a4f-0264c8c19f0f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:48.863039 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a335430c-422d-4c8b-8a4f-0264c8c19f0f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:48.863304 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a335430c-422d-4c8b-8a4f-0264c8c19f0f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:48.872908 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a335430c-422d-4c8b-8a4f-0264c8c19f0f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:48.874389 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a335430c-422d-4c8b-8a4f-0264c8c19f0f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:48.886365 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-a335430c-422d-4c8b-8a4f-0264c8c19f0f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 28 15:28:48.886365 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:48.886365 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:48.886365 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:48.886365 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:48.886365 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:48.886365 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:48.886365 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:48.886365 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:48.886365 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:48.886365 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:48.886365 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:48.886365 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:28:48.886365 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:28:48.886365 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:28:48.886365 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:28:48.887471 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:48.887471 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:48.887471 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:28:48.887471 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:28:48.887471 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:28:48.887471 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:28:48.887471 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:48.887471 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:48.887471 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:48.887471 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:48.887471 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:48.887471 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:48.887471 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:48.887471 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:48.887471 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:48.887471 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:48.887471 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:48.887471 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:48.888398 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:48.888398 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:48.888398 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:28:48.888398 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:28:48.888398 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:28:48.888398 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:28:48.888398 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:28:48.888398 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:28:48.888398 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 28 15:28:48.888398 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:48.888398 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 296/587] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:28:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:48.888398 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a335430c-422d-4c8b-8a4f-0264c8c19f0f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:48.896996 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f400a90-95ad-4019-86b3-9955a5ce88d0 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:48.932301 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a81ece-1445-4c4c-a6e9-41d47db8c383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:48.933011 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a81ece-1445-4c4c-a6e9-41d47db8c383 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:48.933283 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a81ece-1445-4c4c-a6e9-41d47db8c383 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:48.933549 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a81ece-1445-4c4c-a6e9-41d47db8c383 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:48.934288 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a81ece-1445-4c4c-a6e9-41d47db8c383 None admin] RBAC: Adding query filter params (role.id=342b64fe0cdb4925b23c6cd461a152ac, scope.project.id=3dc6c5f5fcd34922b0d09cd2aaed6cf1, user.id=328051fe7607464f8f5dc0dcc01a817a) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:48.934589 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a81ece-1445-4c4c-a6e9-41d47db8c383 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:48.936203 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a81ece-1445-4c4c-a6e9-41d47db8c383 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:48.949883 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 292/588] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 15:28:48 2022] GET /identity/v3/role_assignments?user.id=328051fe7607464f8f5dc0dcc01a817a&scope.project.id=3dc6c5f5fcd34922b0d09cd2aaed6cf1&role.id=342b64fe0cdb4925b23c6cd461a152ac => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:48.949883 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a81ece-1445-4c4c-a6e9-41d47db8c383 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:48.962392 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:49.006705 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a335430c-422d-4c8b-8a4f-0264c8c19f0f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:49.049452 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e09594-8333-4cfd-b964-1ffc313d0752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:49.050263 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e09594-8333-4cfd-b964-1ffc313d0752 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:49.050611 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e09594-8333-4cfd-b964-1ffc313d0752 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:49.050954 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e09594-8333-4cfd-b964-1ffc313d0752 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:49.060276 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e09594-8333-4cfd-b964-1ffc313d0752 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:49.061898 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e09594-8333-4cfd-b964-1ffc313d0752 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:49.074795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-e1e09594-8333-4cfd-b964-1ffc313d0752 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 28 15:28:49.074795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:49.074795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:49.074795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:49.074795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:49.074795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:49.074795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:49.074795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:49.074795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:49.074795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:49.074795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:49.074795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:49.074795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:28:49.074795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:28:49.074795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:28:49.074795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:28:49.075854 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:49.075854 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:49.075854 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:28:49.075854 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:28:49.075854 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:28:49.075854 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:28:49.075854 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:49.075854 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:49.075854 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:49.075854 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:49.075854 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:49.075854 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:49.075854 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:49.075854 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:49.075854 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:49.075854 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:49.075854 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:49.075854 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:49.076752 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:49.076752 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:49.076752 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:28:49.076752 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:28:49.076752 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:28:49.076752 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:28:49.076752 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:28:49.076752 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:28:49.076752 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 28 15:28:49.076752 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:49.080981 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 297/589] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:28:48 2022] GET /identity/v3/users/glance => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:49.081829 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e09594-8333-4cfd-b964-1ffc313d0752 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:49.084420 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a81ece-1445-4c4c-a6e9-41d47db8c383 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:49.110264 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:49.133162 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651ce64b-c811-462c-af9b-498169a38a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:49.134448 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ce64b-c811-462c-af9b-498169a38a91 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:49.134729 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ce64b-c811-462c-af9b-498169a38a91 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:49.135000 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ce64b-c811-462c-af9b-498169a38a91 None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:49.135739 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651ce64b-c811-462c-af9b-498169a38a91 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:49.136041 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651ce64b-c811-462c-af9b-498169a38a91 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:49.137921 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651ce64b-c811-462c-af9b-498169a38a91 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:49.159837 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 293/590] 10.222.0.52 () {56 vars in 1139 bytes} [Wed Sep 28 15:28:48 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:49.160953 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651ce64b-c811-462c-af9b-498169a38a91 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:49.218233 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e09594-8333-4cfd-b964-1ffc313d0752 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:49.250893 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:49.268072 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc68eae-ac04-49a3-a4c8-c1ee82086956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:49.268682 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc68eae-ac04-49a3-a4c8-c1ee82086956 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:49.268969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc68eae-ac04-49a3-a4c8-c1ee82086956 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:49.269247 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc68eae-ac04-49a3-a4c8-c1ee82086956 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:49.269998 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc68eae-ac04-49a3-a4c8-c1ee82086956 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:49.270308 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc68eae-ac04-49a3-a4c8-c1ee82086956 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:49.273530 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc68eae-ac04-49a3-a4c8-c1ee82086956 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:49.293359 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651ce64b-c811-462c-af9b-498169a38a91 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:49.294615 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 298/591] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Sep 28 15:28:49 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:49.295562 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc68eae-ac04-49a3-a4c8-c1ee82086956 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:49.332592 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090b1154-71a4-473c-82da-1efb032912e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:49.333537 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b1154-71a4-473c-82da-1efb032912e1 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:49.334090 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b1154-71a4-473c-82da-1efb032912e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:49.334552 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b1154-71a4-473c-82da-1efb032912e1 None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:49.335574 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090b1154-71a4-473c-82da-1efb032912e1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:49.336089 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090b1154-71a4-473c-82da-1efb032912e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:49.338336 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090b1154-71a4-473c-82da-1efb032912e1 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:49.361926 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 294/592] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Sep 28 15:28:49 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:49.363082 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090b1154-71a4-473c-82da-1efb032912e1 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:49.412397 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:49.419594 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc68eae-ac04-49a3-a4c8-c1ee82086956 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:49.456021 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea03f56-2687-44de-b8f1-9cd53c568d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:49.457095 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea03f56-2687-44de-b8f1-9cd53c568d3d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:49.457612 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea03f56-2687-44de-b8f1-9cd53c568d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:49.458122 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea03f56-2687-44de-b8f1-9cd53c568d3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:49.468969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea03f56-2687-44de-b8f1-9cd53c568d3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:49.470793 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea03f56-2687-44de-b8f1-9cd53c568d3d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:49.487509 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-2ea03f56-2687-44de-b8f1-9cd53c568d3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:49.487509 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:49.487509 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:49.487509 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:49.487509 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:49.487509 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:49.487509 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:49.487509 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:49.487509 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:49.487509 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:49.487509 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:49.487509 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:49.487509 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:28:49.487509 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:28:49.487509 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:28:49.487509 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:28:49.488886 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:49.488886 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:49.488886 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:49.488886 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:49.488886 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:49.488886 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:49.488886 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:49.488886 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:49.488886 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:49.488886 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:49.488886 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:49.488886 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:49.488886 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:49.488886 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:49.488886 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:49.488886 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:49.488886 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:28:49.489772 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:28:49.489772 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:49.489772 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:49.491678 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 299/593] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:49.492649 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea03f56-2687-44de-b8f1-9cd53c568d3d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:49.502807 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090b1154-71a4-473c-82da-1efb032912e1 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:49.530177 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:49.552125 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f905abe-8743-4c0c-ad5a-6210e3bcc276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:49.552758 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f905abe-8743-4c0c-ad5a-6210e3bcc276 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:49.553036 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f905abe-8743-4c0c-ad5a-6210e3bcc276 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:49.553305 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f905abe-8743-4c0c-ad5a-6210e3bcc276 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:49.567823 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f905abe-8743-4c0c-ad5a-6210e3bcc276 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:49.569890 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f905abe-8743-4c0c-ad5a-6210e3bcc276 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:49.590035 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-1f905abe-8743-4c0c-ad5a-6210e3bcc276 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:49.590035 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:49.590035 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:49.590035 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:49.590035 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:49.590035 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:49.590035 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:49.590035 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:49.590035 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:49.590035 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:49.590035 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:49.590035 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:49.590035 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:28:49.590035 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:28:49.590035 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:28:49.590035 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:28:49.591416 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:49.591416 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:49.591416 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:49.591416 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:49.591416 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:49.591416 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:49.591416 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:49.591416 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:49.591416 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:49.591416 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:49.591416 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:49.591416 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:49.591416 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:49.591416 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:49.591416 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:49.591416 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:49.591416 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:28:49.592282 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:28:49.592282 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:49.592282 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:49.592282 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 295/594] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:49.592282 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f905abe-8743-4c0c-ad5a-6210e3bcc276 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:49.609682 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea03f56-2687-44de-b8f1-9cd53c568d3d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:49.654163 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce644c2-0adb-4318-afae-ffb13100689c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:49.655159 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce644c2-0adb-4318-afae-ffb13100689c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:49.655633 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce644c2-0adb-4318-afae-ffb13100689c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:49.656097 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce644c2-0adb-4318-afae-ffb13100689c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:49.667892 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce644c2-0adb-4318-afae-ffb13100689c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:49.669703 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce644c2-0adb-4318-afae-ffb13100689c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:49.684324 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-9ce644c2-0adb-4318-afae-ffb13100689c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:49.684324 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:49.684324 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:49.684324 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:49.684324 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:49.684324 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:49.684324 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:49.684324 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:49.684324 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:49.684324 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:49.684324 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:49.684324 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:49.684324 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:28:49.684324 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:28:49.684324 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:28:49.684324 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:28:49.685267 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:49.685267 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:49.685267 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:49.685267 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:49.685267 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:49.685267 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:49.685267 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:49.685267 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:49.685267 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:49.685267 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:49.685267 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:49.685267 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:49.685267 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:49.685267 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:49.685267 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:49.685267 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:49.685267 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:28:49.686190 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:28:49.686190 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:49.686190 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:49.688229 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 300/595] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:49.689310 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce644c2-0adb-4318-afae-ffb13100689c None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:49.717385 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:49.741501 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f905abe-8743-4c0c-ad5a-6210e3bcc276 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:49.791495 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d715df-c321-49af-bffe-318b9e325ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:49.792110 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d715df-c321-49af-bffe-318b9e325ed4 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:49.792389 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d715df-c321-49af-bffe-318b9e325ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:49.792659 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d715df-c321-49af-bffe-318b9e325ed4 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:49.793558 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d715df-c321-49af-bffe-318b9e325ed4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:49.793881 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d715df-c321-49af-bffe-318b9e325ed4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:49.795054 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:49.796378 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d715df-c321-49af-bffe-318b9e325ed4 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:49.817170 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 296/596] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:28:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:49.817170 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d715df-c321-49af-bffe-318b9e325ed4 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:49.818199 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce644c2-0adb-4318-afae-ffb13100689c None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:49.865517 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d662926-3c84-46db-9ad3-d02dadf782df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:49.866090 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d662926-3c84-46db-9ad3-d02dadf782df None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:49.866375 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d662926-3c84-46db-9ad3-d02dadf782df None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:49.866650 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d662926-3c84-46db-9ad3-d02dadf782df None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:49.867510 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d662926-3c84-46db-9ad3-d02dadf782df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:49.868464 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d662926-3c84-46db-9ad3-d02dadf782df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:49.870932 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d662926-3c84-46db-9ad3-d02dadf782df None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:49.889437 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 301/597] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:28:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:49.890393 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d662926-3c84-46db-9ad3-d02dadf782df None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:49.954448 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d715df-c321-49af-bffe-318b9e325ed4 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:50.005278 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc55c3cd-8ce5-4fe3-a697-93a6d15bff44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:50.006959 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55c3cd-8ce5-4fe3-a697-93a6d15bff44 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:50.007684 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55c3cd-8ce5-4fe3-a697-93a6d15bff44 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:50.008173 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc55c3cd-8ce5-4fe3-a697-93a6d15bff44 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:50.010279 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc55c3cd-8ce5-4fe3-a697-93a6d15bff44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:50.011098 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc55c3cd-8ce5-4fe3-a697-93a6d15bff44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:50.013471 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc55c3cd-8ce5-4fe3-a697-93a6d15bff44 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:50.015980 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d662926-3c84-46db-9ad3-d02dadf782df None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:50.028279 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 297/598] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:28:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:50.029990 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc55c3cd-8ce5-4fe3-a697-93a6d15bff44 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:50.042316 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:50.061173 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f079ce27-8609-47ea-ac2c-a6c53e2be9ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:50.062282 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f079ce27-8609-47ea-ac2c-a6c53e2be9ab None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:50.062577 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f079ce27-8609-47ea-ac2c-a6c53e2be9ab None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:50.062863 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f079ce27-8609-47ea-ac2c-a6c53e2be9ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:50.074122 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f079ce27-8609-47ea-ac2c-a6c53e2be9ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:50.076648 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f079ce27-8609-47ea-ac2c-a6c53e2be9ab None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:50.092994 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-f079ce27-8609-47ea-ac2c-a6c53e2be9ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:28:50.092994 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:50.092994 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:50.092994 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:50.092994 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:50.092994 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:50.092994 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:50.092994 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:50.092994 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:50.092994 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:50.092994 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:50.092994 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:50.092994 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:28:50.092994 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:28:50.092994 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:28:50.094381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:28:50.094381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:50.094381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:50.094381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:50.094381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:50.094381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:50.094381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:50.094381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:50.094381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:50.094381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:50.094381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:50.094381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:50.094381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:50.094381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:50.094381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:50.094381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:50.094381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:50.095524 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:28:50.095524 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:28:50.095524 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:28:50.095524 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:28:50.095524 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:28:50.095524 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:28:50.095524 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:28:50.095524 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:50.102491 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 302/599] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:28:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:50.103517 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f079ce27-8609-47ea-ac2c-a6c53e2be9ab None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:50.112942 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:50.156304 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc55c3cd-8ce5-4fe3-a697-93a6d15bff44 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:50.195470 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd1f04-ceba-4a82-a0c4-c8d23662cdd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:50.197426 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd1f04-ceba-4a82-a0c4-c8d23662cdd3 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:50.197896 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd1f04-ceba-4a82-a0c4-c8d23662cdd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:50.198599 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd1f04-ceba-4a82-a0c4-c8d23662cdd3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:50.207466 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cd1f04-ceba-4a82-a0c4-c8d23662cdd3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:50.209837 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cd1f04-ceba-4a82-a0c4-c8d23662cdd3 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:50.218122 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f079ce27-8609-47ea-ac2c-a6c53e2be9ab None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:50.225523 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-39cd1f04-ceba-4a82-a0c4-c8d23662cdd3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:28:50.225523 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:50.225523 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:50.225523 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:50.225523 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:50.225523 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:50.225523 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:50.225523 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:50.225523 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:50.225523 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:50.225523 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:50.225523 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:50.225523 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:28:50.225523 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:28:50.225523 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:28:50.226676 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:28:50.226676 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:50.226676 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:50.226676 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:50.226676 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:50.226676 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:50.226676 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:50.226676 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:50.226676 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:50.226676 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:50.226676 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:50.226676 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:50.226676 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:50.226676 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:50.226676 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:50.226676 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:50.226676 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:50.227805 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:28:50.227805 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:28:50.227805 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:28:50.227805 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:28:50.227805 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:28:50.227805 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:28:50.227805 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:28:50.227805 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:50.230335 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 298/600] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:28:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:50.231778 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd1f04-ceba-4a82-a0c4-c8d23662cdd3 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:50.261092 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc61ac9-6b86-417d-9d92-52a193e5a871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:50.261944 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc61ac9-6b86-417d-9d92-52a193e5a871 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:50.262335 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc61ac9-6b86-417d-9d92-52a193e5a871 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:50.262715 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc61ac9-6b86-417d-9d92-52a193e5a871 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:50.274317 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc61ac9-6b86-417d-9d92-52a193e5a871 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:50.274317 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc61ac9-6b86-417d-9d92-52a193e5a871 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:50.292975 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-fdc61ac9-6b86-417d-9d92-52a193e5a871 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:28:50.292975 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:50.292975 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:50.292975 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:50.292975 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:50.292975 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:50.292975 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:50.292975 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:50.292975 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:50.292975 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:50.292975 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:50.292975 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:50.292975 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:28:50.292975 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:28:50.292975 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:28:50.293982 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:28:50.293982 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:50.293982 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:50.293982 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:50.293982 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:50.293982 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:50.293982 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:50.293982 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:50.293982 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:50.293982 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:50.293982 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:50.293982 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:50.293982 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:50.293982 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:50.293982 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:50.293982 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:50.293982 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:50.294895 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:28:50.294895 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:28:50.294895 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:28:50.294895 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:28:50.294895 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:28:50.294895 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:28:50.294895 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:28:50.294895 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:50.294895 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 303/601] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:28:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:50.294895 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc61ac9-6b86-417d-9d92-52a193e5a871 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:50.334457 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:50.382295 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:50.384206 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd1f04-ceba-4a82-a0c4-c8d23662cdd3 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:50.426187 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc61ac9-6b86-417d-9d92-52a193e5a871 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:50.427194 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2036f9-d6dc-4514-9cdc-fc6de4da73be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:50.427801 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2036f9-d6dc-4514-9cdc-fc6de4da73be None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:50.428080 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2036f9-d6dc-4514-9cdc-fc6de4da73be None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:50.428351 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2036f9-d6dc-4514-9cdc-fc6de4da73be None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:50.429039 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2036f9-d6dc-4514-9cdc-fc6de4da73be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:50.429342 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2036f9-d6dc-4514-9cdc-fc6de4da73be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:50.430812 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2036f9-d6dc-4514-9cdc-fc6de4da73be None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:50.442675 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 299/602] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 15:28:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:50.443624 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2036f9-d6dc-4514-9cdc-fc6de4da73be None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:50.467821 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d22de6b-70ab-43d1-8347-d74653444c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:50.468432 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22de6b-70ab-43d1-8347-d74653444c29 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:50.468710 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22de6b-70ab-43d1-8347-d74653444c29 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:50.468976 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22de6b-70ab-43d1-8347-d74653444c29 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:50.469654 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d22de6b-70ab-43d1-8347-d74653444c29 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:50.470105 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d22de6b-70ab-43d1-8347-d74653444c29 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:50.471638 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d22de6b-70ab-43d1-8347-d74653444c29 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:50.484726 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 304/603] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 15:28:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:50.486127 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f9cac-afe6-49d9-ba18-cdd7824a8505 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:50.486414 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f9cac-afe6-49d9-ba18-cdd7824a8505 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:50.486687 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f9cac-afe6-49d9-ba18-cdd7824a8505 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:50.488015 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 305/604] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:50.488598 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59f9cac-afe6-49d9-ba18-cdd7824a8505 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:50.554427 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2036f9-d6dc-4514-9cdc-fc6de4da73be None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:50.583095 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:50.592923 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35807781-8b24-4682-bbbc-6713b423d581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:50.593512 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35807781-8b24-4682-bbbc-6713b423d581 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:50.593783 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35807781-8b24-4682-bbbc-6713b423d581 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:50.594082 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35807781-8b24-4682-bbbc-6713b423d581 None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:50.594769 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35807781-8b24-4682-bbbc-6713b423d581 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:50.595073 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35807781-8b24-4682-bbbc-6713b423d581 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:50.596601 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35807781-8b24-4682-bbbc-6713b423d581 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:50.603111 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59f9cac-afe6-49d9-ba18-cdd7824a8505 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:50.613298 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 300/605] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 15:28:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:50.613298 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35807781-8b24-4682-bbbc-6713b423d581 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:50.636122 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:50.647226 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cca0ff6-8a24-4f7b-ab91-a7e9fe0bc2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:50.648149 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca0ff6-8a24-4f7b-ab91-a7e9fe0bc2ed None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:50.648433 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca0ff6-8a24-4f7b-ab91-a7e9fe0bc2ed None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:50.648709 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca0ff6-8a24-4f7b-ab91-a7e9fe0bc2ed None admin] PATH_INFO: `/v3/projects/3dc6c5f5fcd34922b0d09cd2aaed6cf1/users/b13b0533444a449ead00a389c5f6bd1a/roles/342b64fe0cdb4925b23c6cd461a152ac` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:50.667567 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cca0ff6-8a24-4f7b-ab91-a7e9fe0bc2ed None admin] RBAC: Authorizing `identity:create_grant(project_id=3dc6c5f5fcd34922b0d09cd2aaed6cf1, user_id=b13b0533444a449ead00a389c5f6bd1a, role_id=342b64fe0cdb4925b23c6cd461a152ac)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:50.669423 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cca0ff6-8a24-4f7b-ab91-a7e9fe0bc2ed None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:50.686889 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 306/606] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 15:28:50 2022] PUT /identity/v3/projects/3dc6c5f5fcd34922b0d09cd2aaed6cf1/users/b13b0533444a449ead00a389c5f6bd1a/roles/342b64fe0cdb4925b23c6cd461a152ac => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:28:50.689439 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507d02c-928b-4508-ae3d-351f6cedf91d None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:50.689984 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507d02c-928b-4508-ae3d-351f6cedf91d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:50.690444 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507d02c-928b-4508-ae3d-351f6cedf91d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:50.719529 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-3507d02c-928b-4508-ae3d-351f6cedf91d None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:50.721365 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3507d02c-928b-4508-ae3d-351f6cedf91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:50.726462 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35807781-8b24-4682-bbbc-6713b423d581 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:50.768101 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298c9e86-9e13-404b-a20a-abe6cff6b4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:50.768783 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298c9e86-9e13-404b-a20a-abe6cff6b4eb None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:50.769061 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298c9e86-9e13-404b-a20a-abe6cff6b4eb None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:50.769336 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298c9e86-9e13-404b-a20a-abe6cff6b4eb None admin] PATH_INFO: `/v3/projects/3dc6c5f5fcd34922b0d09cd2aaed6cf1/users/01833709177e4e68a2fa52fb9963d3bf/roles/342b64fe0cdb4925b23c6cd461a152ac` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:50.795305 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298c9e86-9e13-404b-a20a-abe6cff6b4eb None admin] RBAC: Authorizing `identity:create_grant(project_id=3dc6c5f5fcd34922b0d09cd2aaed6cf1, user_id=01833709177e4e68a2fa52fb9963d3bf, role_id=342b64fe0cdb4925b23c6cd461a152ac)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:50.796629 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298c9e86-9e13-404b-a20a-abe6cff6b4eb None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:50.817394 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 301/607] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 15:28:50 2022] PUT /identity/v3/projects/3dc6c5f5fcd34922b0d09cd2aaed6cf1/users/01833709177e4e68a2fa52fb9963d3bf/roles/342b64fe0cdb4925b23c6cd461a152ac => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:28:50.818474 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298c9e86-9e13-404b-a20a-abe6cff6b4eb None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:50.818833 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 307/608] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:50.861827 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:50.937888 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298c9e86-9e13-404b-a20a-abe6cff6b4eb None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:50.977840 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a3312-599e-4e9f-ad83-6e590b3fa92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:50.978594 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a3312-599e-4e9f-ad83-6e590b3fa92e None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:50.978939 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a3312-599e-4e9f-ad83-6e590b3fa92e None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:50.979208 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a3312-599e-4e9f-ad83-6e590b3fa92e None admin] PATH_INFO: `/v3/projects/3dc6c5f5fcd34922b0d09cd2aaed6cf1/users/c6a46bbd249c4434ad7d9442754f098a/roles/342b64fe0cdb4925b23c6cd461a152ac` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:50.997677 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183a3312-599e-4e9f-ad83-6e590b3fa92e None admin] RBAC: Authorizing `identity:create_grant(project_id=3dc6c5f5fcd34922b0d09cd2aaed6cf1, user_id=c6a46bbd249c4434ad7d9442754f098a, role_id=342b64fe0cdb4925b23c6cd461a152ac)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:50.999222 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183a3312-599e-4e9f-ad83-6e590b3fa92e None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:51.016293 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 302/609] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 15:28:50 2022] PUT /identity/v3/projects/3dc6c5f5fcd34922b0d09cd2aaed6cf1/users/c6a46bbd249c4434ad7d9442754f098a/roles/342b64fe0cdb4925b23c6cd461a152ac => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:28:51.175715 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3507d02c-928b-4508-ae3d-351f6cedf91d None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:51.206797 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb582c-bf92-4fa8-8ce6-3f28fcbc4fed None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:51.206985 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb582c-bf92-4fa8-8ce6-3f28fcbc4fed None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:51.207246 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb582c-bf92-4fa8-8ce6-3f28fcbc4fed None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:51.208884 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 303/610] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:51.215201 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759b3295-a1de-4ad4-987c-647e0d005330 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:51.215521 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759b3295-a1de-4ad4-987c-647e0d005330 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:51.215735 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759b3295-a1de-4ad4-987c-647e0d005330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:51.235296 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-759b3295-a1de-4ad4-987c-647e0d005330 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:51.236494 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-759b3295-a1de-4ad4-987c-647e0d005330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:51.320766 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3507d02c-928b-4508-ae3d-351f6cedf91d None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:51.329166 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:51.329337 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 304/611] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:28:51.387483 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f8313b-d9e1-4bc3-ab9c-3e88098a00db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:51.388126 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8313b-d9e1-4bc3-ab9c-3e88098a00db None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:51.388401 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8313b-d9e1-4bc3-ab9c-3e88098a00db None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:51.388672 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8313b-d9e1-4bc3-ab9c-3e88098a00db None admin] PATH_INFO: `/v3/roles/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:51.406115 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f8313b-d9e1-4bc3-ab9c-3e88098a00db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:51.410962 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-90f8313b-d9e1-4bc3-ab9c-3e88098a00db None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:28:51.410962 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:51.410962 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:51.410962 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:51.410962 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:51.410962 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:51.410962 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:51.410962 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:51.410962 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:51.410962 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:51.410962 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:51.410962 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:51.410962 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:51.410962 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:51.410962 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:51.410962 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:51.411875 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:51.411875 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:51.411875 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:51.411875 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:51.411875 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:51.411875 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:51.411875 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:51.411875 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:51.411875 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:51.411875 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:51.411875 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:51.411875 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:51.411875 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:51.411875 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:51.411875 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:51.411875 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:51.412702 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:51.412702 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:51.412702 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:51.412702 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:51.412702 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:51.412702 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:51.412702 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:51.412702 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:51.412702 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:51.412702 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:51.412702 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:28:51.412702 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:51.412702 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 308/612] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 15:28:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:51.420149 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759b3295-a1de-4ad4-987c-647e0d005330 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:51.444013 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:51.546863 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759b3295-a1de-4ad4-987c-647e0d005330 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:51.586073 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da981f4e-98e5-4edc-b04e-4df17ac636c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:51.587406 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da981f4e-98e5-4edc-b04e-4df17ac636c6 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:51.587682 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da981f4e-98e5-4edc-b04e-4df17ac636c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:51.587948 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da981f4e-98e5-4edc-b04e-4df17ac636c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:51.589087 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da981f4e-98e5-4edc-b04e-4df17ac636c6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:51.589394 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da981f4e-98e5-4edc-b04e-4df17ac636c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:51.590779 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da981f4e-98e5-4edc-b04e-4df17ac636c6 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:51.599551 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 305/613] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 15:28:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:51.608251 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f8313b-d9e1-4bc3-ab9c-3e88098a00db None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:51.684951 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da981f4e-98e5-4edc-b04e-4df17ac636c6 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:51.802222 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:51.812138 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f8313b-d9e1-4bc3-ab9c-3e88098a00db None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:51.841728 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da981f4e-98e5-4edc-b04e-4df17ac636c6 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:51.878666 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080d414e-b9e3-495f-8563-26458bb55827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:51.879310 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d414e-b9e3-495f-8563-26458bb55827 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:51.879583 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d414e-b9e3-495f-8563-26458bb55827 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:51.880069 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d414e-b9e3-495f-8563-26458bb55827 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:51.892915 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080d414e-b9e3-495f-8563-26458bb55827 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:51.894436 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080d414e-b9e3-495f-8563-26458bb55827 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:51.910196 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-080d414e-b9e3-495f-8563-26458bb55827 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:51.910196 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:51.910196 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:51.910196 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:51.910196 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:51.910196 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:51.910196 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:51.910196 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:51.910196 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:51.910196 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:51.910196 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:51.910196 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:51.910196 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:28:51.910196 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:28:51.910196 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:28:51.910196 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:28:51.911192 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:51.911192 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:51.911192 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:51.911192 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:51.911192 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:51.911192 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:51.911192 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:51.911192 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:51.911192 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:51.911192 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:51.911192 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:51.911192 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:51.911192 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:51.911192 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:51.911192 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:51.911192 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:51.911192 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:28:51.912061 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:28:51.912061 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:51.912061 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:51.912061 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 309/614] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:51.918715 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080d414e-b9e3-495f-8563-26458bb55827 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:51.939652 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5813241a-e2e6-4daa-8e89-a11c7019a029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:51.940285 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5813241a-e2e6-4daa-8e89-a11c7019a029 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:51.940561 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5813241a-e2e6-4daa-8e89-a11c7019a029 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:51.940826 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5813241a-e2e6-4daa-8e89-a11c7019a029 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:51.956047 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5813241a-e2e6-4daa-8e89-a11c7019a029 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:51.958185 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:51.962610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-5813241a-e2e6-4daa-8e89-a11c7019a029 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:28:51.962610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:51.962610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:51.962610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:51.962610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:51.962610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:51.962610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:51.962610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:51.962610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:51.962610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:51.962610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:51.962610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:51.962610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:51.962610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:51.962610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:51.962610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:51.963810 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:51.963810 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:51.963810 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:51.963810 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:51.963810 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:51.963810 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:51.963810 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:51.963810 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:51.963810 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:51.963810 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:51.963810 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:51.963810 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:51.963810 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:51.963810 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:51.963810 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:51.963810 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:51.964791 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:51.964791 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:51.964791 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:51.964791 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:51.964791 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:51.964791 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:51.964791 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:51.964791 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:51.964791 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:51.964791 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:51.964791 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:28:51.964791 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:51.964791 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 306/615] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 15:28:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:28:51.975113 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5813241a-e2e6-4daa-8e89-a11c7019a029 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:52.055729 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080d414e-b9e3-495f-8563-26458bb55827 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:52.086058 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5813241a-e2e6-4daa-8e89-a11c7019a029 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:52.104069 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555db02-5629-4477-8273-a77e3a496f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:52.105007 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555db02-5629-4477-8273-a77e3a496f95 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:52.105508 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555db02-5629-4477-8273-a77e3a496f95 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:52.106001 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f555db02-5629-4477-8273-a77e3a496f95 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:52.106979 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555db02-5629-4477-8273-a77e3a496f95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:52.107476 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555db02-5629-4477-8273-a77e3a496f95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:52.109155 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f555db02-5629-4477-8273-a77e3a496f95 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:52.120652 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7630f9-7957-4bd1-8e27-2c27d68e933a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:52.121260 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7630f9-7957-4bd1-8e27-2c27d68e933a None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:52.126299 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7630f9-7957-4bd1-8e27-2c27d68e933a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:52.126589 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7630f9-7957-4bd1-8e27-2c27d68e933a None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:52.127279 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7630f9-7957-4bd1-8e27-2c27d68e933a None admin] RBAC: Adding query filter params (name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:52.127586 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7630f9-7957-4bd1-8e27-2c27d68e933a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:52.128956 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7630f9-7957-4bd1-8e27-2c27d68e933a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:52.129392 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 310/616] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:28:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:52.137234 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:52.138092 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555db02-5629-4477-8273-a77e3a496f95 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:52.143854 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 307/617] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 15:28:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:52.149938 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7630f9-7957-4bd1-8e27-2c27d68e933a None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:52.286060 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:52.297065 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7630f9-7957-4bd1-8e27-2c27d68e933a None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:52.338619 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506c100e-277e-43ee-81ad-52191aabde24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:52.339605 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c100e-277e-43ee-81ad-52191aabde24 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:52.340239 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c100e-277e-43ee-81ad-52191aabde24 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:52.342295 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f555db02-5629-4477-8273-a77e3a496f95 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:52.342798 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c100e-277e-43ee-81ad-52191aabde24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:52.362703 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506c100e-277e-43ee-81ad-52191aabde24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:52.366493 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506c100e-277e-43ee-81ad-52191aabde24 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:52.390632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-506c100e-277e-43ee-81ad-52191aabde24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:52.390632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:52.390632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:52.390632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:52.390632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:52.390632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:52.390632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:52.390632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:52.390632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:52.390632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:52.390632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:52.390632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:52.390632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:28:52.390632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:28:52.390632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:28:52.390632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:28:52.391603 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:52.391603 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:52.391603 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:52.391603 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:52.391603 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:52.391603 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:52.391603 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:52.391603 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:52.391603 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:52.391603 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:52.391603 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:52.391603 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:52.391603 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:52.391603 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:52.391603 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:52.391603 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:52.391603 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:28:52.392554 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:28:52.392554 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:52.392554 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:52.394774 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 308/618] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:28:52.404596 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506c100e-277e-43ee-81ad-52191aabde24 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:52.419652 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ee03fe-bbc2-4830-9cc4-1d8f5f82ac55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:52.420284 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee03fe-bbc2-4830-9cc4-1d8f5f82ac55 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:52.420556 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee03fe-bbc2-4830-9cc4-1d8f5f82ac55 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:52.420820 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee03fe-bbc2-4830-9cc4-1d8f5f82ac55 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:52.432190 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ee03fe-bbc2-4830-9cc4-1d8f5f82ac55 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:52.433629 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ee03fe-bbc2-4830-9cc4-1d8f5f82ac55 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:52.453049 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-26ee03fe-bbc2-4830-9cc4-1d8f5f82ac55 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 28 15:28:52.453049 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:52.453049 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:52.453049 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:52.453049 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:52.453049 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:52.453049 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:52.453049 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:52.453049 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:52.453049 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:52.453049 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:52.453049 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:52.453049 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:28:52.453049 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:28:52.453049 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:28:52.453049 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:28:52.454223 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:52.454223 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:52.454223 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:28:52.454223 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:28:52.454223 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:28:52.454223 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:28:52.454223 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:52.454223 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:52.454223 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:52.454223 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:52.454223 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:52.454223 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:52.454223 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:52.454223 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:52.454223 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:52.454223 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:52.454223 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:52.454223 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:52.455164 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:52.455164 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:52.455164 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:28:52.455164 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:28:52.455164 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:28:52.455164 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:28:52.455164 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:28:52.455164 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:28:52.455164 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 28 15:28:52.455164 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:52.455716 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 311/619] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Sep 28 15:28:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:28:52.463004 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ee03fe-bbc2-4830-9cc4-1d8f5f82ac55 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:52.499974 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:52.573087 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506c100e-277e-43ee-81ad-52191aabde24 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:52.596970 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ee03fe-bbc2-4830-9cc4-1d8f5f82ac55 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:52.614434 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24207015-870f-4be0-9f24-0655d4f2d504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:52.615088 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24207015-870f-4be0-9f24-0655d4f2d504 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:52.615317 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24207015-870f-4be0-9f24-0655d4f2d504 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:52.615589 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24207015-870f-4be0-9f24-0655d4f2d504 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:52.616228 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24207015-870f-4be0-9f24-0655d4f2d504 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:52.616532 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24207015-870f-4be0-9f24-0655d4f2d504 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:52.619189 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24207015-870f-4be0-9f24-0655d4f2d504 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:52.642594 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 309/620] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:28:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:52.648442 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24207015-870f-4be0-9f24-0655d4f2d504 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:52.671531 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d82147-6ee3-4ae1-ad56-432323d618ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:52.672176 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d82147-6ee3-4ae1-ad56-432323d618ea None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:52.672403 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d82147-6ee3-4ae1-ad56-432323d618ea None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:52.672671 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d82147-6ee3-4ae1-ad56-432323d618ea None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:52.673606 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d82147-6ee3-4ae1-ad56-432323d618ea None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:52.673930 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d82147-6ee3-4ae1-ad56-432323d618ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:52.675360 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d82147-6ee3-4ae1-ad56-432323d618ea None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:52.701701 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:52.713597 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 312/621] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Sep 28 15:28:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:52.715080 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d82147-6ee3-4ae1-ad56-432323d618ea None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:52.820170 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d82147-6ee3-4ae1-ad56-432323d618ea None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:52.857495 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7638bbcc-ba24-4c28-a3b5-f9a30db47237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:52.858192 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7638bbcc-ba24-4c28-a3b5-f9a30db47237 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:52.858478 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7638bbcc-ba24-4c28-a3b5-f9a30db47237 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:52.858745 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7638bbcc-ba24-4c28-a3b5-f9a30db47237 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:52.929219 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24207015-870f-4be0-9f24-0655d4f2d504 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:52.938184 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:52.945177 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7638bbcc-ba24-4c28-a3b5-f9a30db47237 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:52.946757 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7638bbcc-ba24-4c28-a3b5-f9a30db47237 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:52.962215 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:52.993928 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51672a5-2326-4add-bce2-13e3c565766d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:52.994561 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51672a5-2326-4add-bce2-13e3c565766d None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:52.995178 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51672a5-2326-4add-bce2-13e3c565766d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:52.995450 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51672a5-2326-4add-bce2-13e3c565766d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:53.003453 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51672a5-2326-4add-bce2-13e3c565766d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:53.004878 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51672a5-2326-4add-bce2-13e3c565766d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:53.017190 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-d51672a5-2326-4add-bce2-13e3c565766d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 28 15:28:53.017190 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:53.017190 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:53.017190 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:53.017190 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:53.017190 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:53.017190 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:53.017190 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:53.017190 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:53.017190 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:53.017190 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:53.017190 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:53.017190 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:28:53.017190 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:28:53.017190 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:28:53.017190 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:28:53.018252 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:53.018252 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:53.018252 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:28:53.018252 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:28:53.018252 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:28:53.018252 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:28:53.018252 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:53.018252 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:53.018252 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:53.018252 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:53.018252 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:53.018252 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:53.018252 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:53.018252 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:53.018252 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:53.018252 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:53.018252 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:53.018252 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:53.019223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:53.019223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:53.019223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:28:53.019223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:28:53.019223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:28:53.019223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:28:53.019223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:28:53.019223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:28:53.019223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 28 15:28:53.019223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:53.022948 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 310/622] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 15:28:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:53.024708 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51672a5-2326-4add-bce2-13e3c565766d None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:53.123711 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51672a5-2326-4add-bce2-13e3c565766d None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:53.158471 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c57eb1b-48a2-4cfb-bde5-8311af75e110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:53.159070 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57eb1b-48a2-4cfb-bde5-8311af75e110 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:53.159343 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57eb1b-48a2-4cfb-bde5-8311af75e110 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:53.159609 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57eb1b-48a2-4cfb-bde5-8311af75e110 None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:53.160320 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c57eb1b-48a2-4cfb-bde5-8311af75e110 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:53.160675 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 313/623] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:28:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:28:53.165149 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c57eb1b-48a2-4cfb-bde5-8311af75e110 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:53.165149 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c57eb1b-48a2-4cfb-bde5-8311af75e110 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:53.166468 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7638bbcc-ba24-4c28-a3b5-f9a30db47237 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:53.191292 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 311/624] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Sep 28 15:28:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:53.196607 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c57eb1b-48a2-4cfb-bde5-8311af75e110 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:53.245630 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:53.271107 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:53.288813 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7638bbcc-ba24-4c28-a3b5-f9a30db47237 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:53.341067 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39474165-3d08-4673-8c1b-98d73fc1bd67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:53.341811 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39474165-3d08-4673-8c1b-98d73fc1bd67 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:53.342174 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39474165-3d08-4673-8c1b-98d73fc1bd67 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:53.342517 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39474165-3d08-4673-8c1b-98d73fc1bd67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:53.347174 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c57eb1b-48a2-4cfb-bde5-8311af75e110 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:53.353781 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39474165-3d08-4673-8c1b-98d73fc1bd67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:53.355360 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39474165-3d08-4673-8c1b-98d73fc1bd67 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:53.372037 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-39474165-3d08-4673-8c1b-98d73fc1bd67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:53.372037 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:53.372037 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:53.372037 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:53.372037 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:53.372037 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:53.372037 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:53.372037 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:53.372037 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:53.372037 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:53.372037 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:53.372037 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:53.372037 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:28:53.372037 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:28:53.372037 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:28:53.372037 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:28:53.373249 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:53.373249 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:53.373249 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:53.373249 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:53.373249 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:53.373249 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:53.373249 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:53.373249 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:53.373249 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:53.373249 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:53.373249 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:53.373249 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:53.373249 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:53.373249 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:53.373249 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:53.373249 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:53.373249 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:28:53.374418 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:28:53.374418 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:53.374418 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:53.379692 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 314/625] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:53.381389 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85867931-2e8e-4531-aa3c-1aa9c58492c9 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:53.381900 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85867931-2e8e-4531-aa3c-1aa9c58492c9 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:53.382192 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85867931-2e8e-4531-aa3c-1aa9c58492c9 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:53.384505 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 315/626] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:53.385331 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85867931-2e8e-4531-aa3c-1aa9c58492c9 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:53.388708 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05b29a-e4c6-4954-b0ce-1b7221ce313e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:53.389887 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05b29a-e4c6-4954-b0ce-1b7221ce313e None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:53.390441 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05b29a-e4c6-4954-b0ce-1b7221ce313e None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:53.390903 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05b29a-e4c6-4954-b0ce-1b7221ce313e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:53.401609 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e05b29a-e4c6-4954-b0ce-1b7221ce313e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:53.403324 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e05b29a-e4c6-4954-b0ce-1b7221ce313e None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:53.418154 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-0e05b29a-e4c6-4954-b0ce-1b7221ce313e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:53.418154 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:53.418154 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:53.418154 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:53.418154 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:53.418154 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:53.418154 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:53.418154 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:53.418154 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:53.418154 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:53.418154 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:53.418154 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:53.418154 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:28:53.418154 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:28:53.418154 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:28:53.418154 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:28:53.419195 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:53.419195 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:53.419195 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:53.419195 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:53.419195 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:53.419195 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:53.419195 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:53.419195 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:53.419195 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:53.419195 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:53.419195 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:53.419195 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:53.419195 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:53.419195 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:53.419195 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:53.419195 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:53.419195 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:28:53.420100 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:28:53.420100 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:53.420100 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:53.426437 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 312/627] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:28:53.427171 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f57937-3b9c-460d-abe2-c5d362d4b724 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:53.427458 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f57937-3b9c-460d-abe2-c5d362d4b724 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:53.427726 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f57937-3b9c-460d-abe2-c5d362d4b724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:53.450785 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-09f57937-3b9c-460d-abe2-c5d362d4b724 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:53.452342 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-09f57937-3b9c-460d-abe2-c5d362d4b724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:53.509950 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85867931-2e8e-4531-aa3c-1aa9c58492c9 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:53.523296 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 313/628] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:53.524680 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f57937-3b9c-460d-abe2-c5d362d4b724 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:53.549686 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:53.584177 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfe59b4-f00a-4c86-8531-1a7df2001337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:53.585042 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe59b4-f00a-4c86-8531-1a7df2001337 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:53.585706 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe59b4-f00a-4c86-8531-1a7df2001337 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:53.587357 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe59b4-f00a-4c86-8531-1a7df2001337 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:53.588170 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfe59b4-f00a-4c86-8531-1a7df2001337 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:53.589154 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfe59b4-f00a-4c86-8531-1a7df2001337 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:53.591629 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfe59b4-f00a-4c86-8531-1a7df2001337 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:53.616602 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 316/629] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:28:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:53.626976 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfe59b4-f00a-4c86-8531-1a7df2001337 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:53.650031 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:53.666856 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f57937-3b9c-460d-abe2-c5d362d4b724 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:53.704688 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1f0c18-32af-49c5-989f-0616d2640e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:53.705290 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f0c18-32af-49c5-989f-0616d2640e68 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:53.705562 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f0c18-32af-49c5-989f-0616d2640e68 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:53.705826 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f0c18-32af-49c5-989f-0616d2640e68 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:53.706476 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1f0c18-32af-49c5-989f-0616d2640e68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:53.706771 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1f0c18-32af-49c5-989f-0616d2640e68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:53.708108 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1f0c18-32af-49c5-989f-0616d2640e68 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:53.719743 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 314/630] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:28:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:53.720966 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78adcc8e-82a1-46b9-bb3e-b100ba56715b None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:53.721245 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78adcc8e-82a1-46b9-bb3e-b100ba56715b None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:53.721511 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78adcc8e-82a1-46b9-bb3e-b100ba56715b None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:53.722857 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 315/631] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:53.723731 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fdef17-ad37-4f38-863d-14374891a527 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:53.724163 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fdef17-ad37-4f38-863d-14374891a527 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:53.724410 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fdef17-ad37-4f38-863d-14374891a527 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:53.729633 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 316/632] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:53.729633 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fdef17-ad37-4f38-863d-14374891a527 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:53.755289 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfe59b4-f00a-4c86-8531-1a7df2001337 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:53.794710 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fbcd32-c847-432c-87f8-7f4afd8a6f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:53.795724 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fbcd32-c847-432c-87f8-7f4afd8a6f99 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:53.796203 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fbcd32-c847-432c-87f8-7f4afd8a6f99 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:53.796636 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fbcd32-c847-432c-87f8-7f4afd8a6f99 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:53.803740 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:53.804393 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fbcd32-c847-432c-87f8-7f4afd8a6f99 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:53.806223 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fbcd32-c847-432c-87f8-7f4afd8a6f99 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:53.818525 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-63fbcd32-c847-432c-87f8-7f4afd8a6f99 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:28:53.818525 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:53.818525 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:53.818525 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:53.818525 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:53.818525 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:53.818525 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:53.818525 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:53.818525 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:53.818525 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:53.818525 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:53.818525 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:53.818525 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:28:53.818525 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:28:53.818525 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:28:53.819461 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:28:53.819461 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:53.819461 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:53.819461 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:53.819461 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:53.819461 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:53.819461 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:53.819461 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:53.819461 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:53.819461 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:53.819461 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:53.819461 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:53.819461 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:53.819461 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:53.819461 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:53.819461 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:53.819461 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:53.820340 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:28:53.820340 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:28:53.820340 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:28:53.820340 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:28:53.820340 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:28:53.820340 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:28:53.820340 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:28:53.820340 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:53.820340 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 317/633] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:28:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:53.821387 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b3593-58e2-4c80-a2a1-1269a6c7a1f4 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:53.822220 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b3593-58e2-4c80-a2a1-1269a6c7a1f4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:53.822220 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b3593-58e2-4c80-a2a1-1269a6c7a1f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:53.840010 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-368b3593-58e2-4c80-a2a1-1269a6c7a1f4 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:53.840943 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fdef17-ad37-4f38-863d-14374891a527 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:53.841195 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-368b3593-58e2-4c80-a2a1-1269a6c7a1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:53.874576 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae84a441-2425-495b-8d5e-1963c4c3ea46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:53.875189 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84a441-2425-495b-8d5e-1963c4c3ea46 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:53.875461 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84a441-2425-495b-8d5e-1963c4c3ea46 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:53.875726 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84a441-2425-495b-8d5e-1963c4c3ea46 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:53.883677 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae84a441-2425-495b-8d5e-1963c4c3ea46 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:53.885040 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae84a441-2425-495b-8d5e-1963c4c3ea46 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:53.896074 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-ae84a441-2425-495b-8d5e-1963c4c3ea46 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:28:53.896074 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:53.896074 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:53.896074 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:53.896074 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:53.896074 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:53.896074 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:53.896074 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:53.896074 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:53.896074 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:53.896074 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:53.896074 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:53.896074 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:28:53.896074 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:28:53.896074 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:28:53.896977 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:28:53.896977 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:53.896977 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:53.896977 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:53.896977 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:53.896977 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:53.896977 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:53.896977 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:53.896977 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:53.896977 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:53.896977 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:53.896977 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:53.896977 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:53.896977 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:53.896977 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:53.896977 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:53.896977 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:53.897871 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:28:53.897871 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:28:53.897871 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:28:53.897871 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:28:53.897871 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:28:53.897871 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:28:53.897871 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:28:53.897871 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:53.897871 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 317/634] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:28:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:53.898362 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b89a09-78ca-4ebc-b993-59731ff9868c None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:53.899566 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b89a09-78ca-4ebc-b993-59731ff9868c None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:53.899566 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b89a09-78ca-4ebc-b993-59731ff9868c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:53.905037 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:53.911401 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 318/635] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:53.912181 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b3593-58e2-4c80-a2a1-1269a6c7a1f4 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:53.919538 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-00b89a09-78ca-4ebc-b993-59731ff9868c None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:53.920695 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-00b89a09-78ca-4ebc-b993-59731ff9868c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:53.997874 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 318/636] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:53.997874 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b89a09-78ca-4ebc-b993-59731ff9868c None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:54.033776 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b3593-58e2-4c80-a2a1-1269a6c7a1f4 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:54.061607 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:54.073274 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e81bf-ddd4-4693-abe3-4792697b1461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:54.073922 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e81bf-ddd4-4693-abe3-4792697b1461 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:54.074203 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e81bf-ddd4-4693-abe3-4792697b1461 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:54.074517 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e81bf-ddd4-4693-abe3-4792697b1461 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:54.075197 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414e81bf-ddd4-4693-abe3-4792697b1461 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:54.075499 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414e81bf-ddd4-4693-abe3-4792697b1461 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:54.077471 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414e81bf-ddd4-4693-abe3-4792697b1461 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:54.094008 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 319/637] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 15:28:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:54.094850 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e81bf-ddd4-4693-abe3-4792697b1461 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:54.107161 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b89a09-78ca-4ebc-b993-59731ff9868c None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:54.144237 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51642728-43eb-4859-838a-2ff55d71c75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:54.144787 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51642728-43eb-4859-838a-2ff55d71c75f None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:54.145063 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51642728-43eb-4859-838a-2ff55d71c75f None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:54.145335 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51642728-43eb-4859-838a-2ff55d71c75f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:54.156383 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51642728-43eb-4859-838a-2ff55d71c75f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:54.161340 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-51642728-43eb-4859-838a-2ff55d71c75f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:28:54.161340 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:54.161340 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:54.161340 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:54.161340 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:54.161340 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:54.161340 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:54.161340 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:54.161340 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:54.161340 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:54.161340 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:54.161340 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:54.161340 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:54.161340 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:54.161340 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:54.161340 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:54.162318 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:54.162318 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:54.162318 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:54.162318 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:54.162318 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:54.162318 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:54.162318 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:54.162318 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:54.162318 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:54.162318 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:54.162318 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:54.162318 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:54.162318 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:54.162318 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:54.162318 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:54.162318 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:54.163141 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:54.163141 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:54.163141 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:54.163141 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:54.163141 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:54.163141 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:54.163141 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:54.163141 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:54.163141 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:54.163141 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:54.163141 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:28:54.163141 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:54.163141 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 319/638] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 15:28:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:54.165064 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51642728-43eb-4859-838a-2ff55d71c75f None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:54.166049 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:54.212643 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e81bf-ddd4-4693-abe3-4792697b1461 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:54.246543 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2456be-c9a4-42e4-b57b-dcf3b72d3fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:54.247149 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2456be-c9a4-42e4-b57b-dcf3b72d3fbe None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:54.247424 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2456be-c9a4-42e4-b57b-dcf3b72d3fbe None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:54.247694 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2456be-c9a4-42e4-b57b-dcf3b72d3fbe None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:54.248374 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2456be-c9a4-42e4-b57b-dcf3b72d3fbe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:54.248669 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2456be-c9a4-42e4-b57b-dcf3b72d3fbe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:54.250117 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2456be-c9a4-42e4-b57b-dcf3b72d3fbe None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:54.267311 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 320/639] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 15:28:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:54.267311 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2456be-c9a4-42e4-b57b-dcf3b72d3fbe None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:54.280051 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51642728-43eb-4859-838a-2ff55d71c75f None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:54.303263 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:54.314926 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412bb40c-43bc-4192-8205-f4c9d1ffe55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:54.315518 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412bb40c-43bc-4192-8205-f4c9d1ffe55d None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:54.315788 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412bb40c-43bc-4192-8205-f4c9d1ffe55d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:54.316051 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412bb40c-43bc-4192-8205-f4c9d1ffe55d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:54.316759 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412bb40c-43bc-4192-8205-f4c9d1ffe55d None admin] RBAC: Adding query filter params (role.id=342b64fe0cdb4925b23c6cd461a152ac, scope.project.id=3dc6c5f5fcd34922b0d09cd2aaed6cf1, user.id=8daf14b64bf04b2ba74d2fe1fd41ff71) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:54.317055 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412bb40c-43bc-4192-8205-f4c9d1ffe55d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:54.318504 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412bb40c-43bc-4192-8205-f4c9d1ffe55d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:54.324744 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 320/640] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 15:28:54 2022] GET /identity/v3/role_assignments?user.id=8daf14b64bf04b2ba74d2fe1fd41ff71&scope.project.id=3dc6c5f5fcd34922b0d09cd2aaed6cf1&role.id=342b64fe0cdb4925b23c6cd461a152ac => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:54.325486 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412bb40c-43bc-4192-8205-f4c9d1ffe55d None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:54.375902 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2456be-c9a4-42e4-b57b-dcf3b72d3fbe None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:54.397997 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:54.422385 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f02d5-b39f-4246-a35f-3512e0bd7377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:54.423287 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f02d5-b39f-4246-a35f-3512e0bd7377 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:54.423570 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f02d5-b39f-4246-a35f-3512e0bd7377 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:54.423841 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5f02d5-b39f-4246-a35f-3512e0bd7377 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:54.424784 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5f02d5-b39f-4246-a35f-3512e0bd7377 None admin] RBAC: Adding query filter params (name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:54.425478 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5f02d5-b39f-4246-a35f-3512e0bd7377 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:54.426918 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5f02d5-b39f-4246-a35f-3512e0bd7377 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:54.437607 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 321/641] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 15:28:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:54.438805 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f02d5-b39f-4246-a35f-3512e0bd7377 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:54.446728 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412bb40c-43bc-4192-8205-f4c9d1ffe55d None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:54.484673 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb738fc6-b49d-4fe1-8636-8e705f73f940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:54.485308 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb738fc6-b49d-4fe1-8636-8e705f73f940 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:54.485582 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb738fc6-b49d-4fe1-8636-8e705f73f940 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:54.485875 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb738fc6-b49d-4fe1-8636-8e705f73f940 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:54.498134 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb738fc6-b49d-4fe1-8636-8e705f73f940 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:54.503478 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-bb738fc6-b49d-4fe1-8636-8e705f73f940 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:28:54.503478 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:54.503478 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:54.503478 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:54.503478 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:54.503478 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:54.503478 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:54.503478 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:54.503478 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:54.503478 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:54.503478 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:54.503478 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:54.503478 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:54.503478 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:54.503478 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:54.503478 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:54.504411 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:54.504411 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:54.504411 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:54.504411 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:54.504411 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:54.504411 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:54.504411 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:54.504411 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:54.504411 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:54.504411 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:54.504411 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:54.504411 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:54.504411 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:54.504411 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:54.504411 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:54.504411 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:54.505246 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:54.505246 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:54.505246 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:54.505246 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:54.505246 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:54.505246 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:54.505246 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:54.505246 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:54.505246 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:54.505246 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:54.505246 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:28:54.505246 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:54.505246 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 321/642] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 15:28:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:54.506638 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb738fc6-b49d-4fe1-8636-8e705f73f940 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:54.555717 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:54.562361 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5f02d5-b39f-4246-a35f-3512e0bd7377 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:54.603637 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b99b80-3a14-4303-b5d1-e986302aa674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:54.604841 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b99b80-3a14-4303-b5d1-e986302aa674 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:54.605119 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b99b80-3a14-4303-b5d1-e986302aa674 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:54.605418 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b99b80-3a14-4303-b5d1-e986302aa674 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:54.613551 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb738fc6-b49d-4fe1-8636-8e705f73f940 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:54.615328 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b99b80-3a14-4303-b5d1-e986302aa674 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:54.621979 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-98b99b80-3a14-4303-b5d1-e986302aa674 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:28:54.621979 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:54.621979 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:54.621979 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:54.621979 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:54.621979 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:54.621979 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:54.621979 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:54.621979 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:54.621979 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:54.621979 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:54.621979 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:54.621979 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:54.621979 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:54.621979 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:54.621979 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:54.623134 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:54.623134 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:54.623134 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:54.623134 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:54.623134 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:54.623134 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:54.623134 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:54.623134 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:54.623134 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:54.623134 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:54.623134 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:54.623134 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:54.623134 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:54.623134 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:54.623134 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:54.623134 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:54.624125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:54.624125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:54.624125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:54.624125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:54.624125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:54.624125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:54.624125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:54.624125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:54.624125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:54.624125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:54.624125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:28:54.624125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:54.624125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 322/643] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 15:28:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:54.624125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b99b80-3a14-4303-b5d1-e986302aa674 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:54.657673 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e75fb64-231b-40bc-a0b1-da0d16b41100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:54.658619 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75fb64-231b-40bc-a0b1-da0d16b41100 None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:54.659176 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75fb64-231b-40bc-a0b1-da0d16b41100 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:54.660373 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75fb64-231b-40bc-a0b1-da0d16b41100 None admin] PATH_INFO: `/v3/projects/3dc6c5f5fcd34922b0d09cd2aaed6cf1/users/328051fe7607464f8f5dc0dcc01a817a/roles/342b64fe0cdb4925b23c6cd461a152ac` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:54.667316 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:54.675966 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e75fb64-231b-40bc-a0b1-da0d16b41100 None admin] RBAC: Authorizing `identity:create_grant(project_id=3dc6c5f5fcd34922b0d09cd2aaed6cf1, user_id=328051fe7607464f8f5dc0dcc01a817a, role_id=342b64fe0cdb4925b23c6cd461a152ac)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:54.677687 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e75fb64-231b-40bc-a0b1-da0d16b41100 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:54.701997 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 322/644] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 15:28:54 2022] PUT /identity/v3/projects/3dc6c5f5fcd34922b0d09cd2aaed6cf1/users/328051fe7607464f8f5dc0dcc01a817a/roles/342b64fe0cdb4925b23c6cd461a152ac => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:28:54.701997 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e75fb64-231b-40bc-a0b1-da0d16b41100 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:54.745104 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b99b80-3a14-4303-b5d1-e986302aa674 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:54.787730 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c4cb7-ed8a-4c0c-8db8-ed5e12724bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:54.788531 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c4cb7-ed8a-4c0c-8db8-ed5e12724bf3 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:54.788709 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c4cb7-ed8a-4c0c-8db8-ed5e12724bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:54.789207 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c4cb7-ed8a-4c0c-8db8-ed5e12724bf3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:54.802514 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9c4cb7-ed8a-4c0c-8db8-ed5e12724bf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:54.804242 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9c4cb7-ed8a-4c0c-8db8-ed5e12724bf3 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:54.821904 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-5b9c4cb7-ed8a-4c0c-8db8-ed5e12724bf3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:54.821904 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:54.821904 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:54.821904 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:54.821904 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:54.821904 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:54.821904 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:54.821904 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:54.821904 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:54.821904 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:54.821904 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:54.821904 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:54.821904 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:28:54.821904 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:28:54.821904 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:28:54.821904 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:28:54.822947 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:54.822947 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:54.822947 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:54.822947 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:54.822947 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:54.822947 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:54.822947 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:54.822947 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:54.822947 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:54.822947 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:54.822947 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:54.822947 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:54.822947 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:54.822947 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:54.822947 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:54.822947 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:54.822947 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:28:54.823998 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:28:54.823998 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:54.823998 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:54.823998 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 323/645] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:54.823998 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c4cb7-ed8a-4c0c-8db8-ed5e12724bf3 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:54.849307 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:54.857511 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e75fb64-231b-40bc-a0b1-da0d16b41100 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:54.904053 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be3ce19-aca9-421a-94a2-6a87b9be8957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:54.905076 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3ce19-aca9-421a-94a2-6a87b9be8957 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:54.905798 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3ce19-aca9-421a-94a2-6a87b9be8957 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:54.906329 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3ce19-aca9-421a-94a2-6a87b9be8957 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:54.907622 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be3ce19-aca9-421a-94a2-6a87b9be8957 None admin] RBAC: Adding query filter params (name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:54.908171 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be3ce19-aca9-421a-94a2-6a87b9be8957 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:54.909733 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be3ce19-aca9-421a-94a2-6a87b9be8957 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:54.920265 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 323/646] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 15:28:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:54.921522 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be3ce19-aca9-421a-94a2-6a87b9be8957 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:54.962534 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:54.972639 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c4cb7-ed8a-4c0c-8db8-ed5e12724bf3 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:55.010062 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3c6215-5f91-4833-94e2-598b2abe84a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:55.010829 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c6215-5f91-4833-94e2-598b2abe84a1 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:55.011198 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c6215-5f91-4833-94e2-598b2abe84a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:55.011955 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c6215-5f91-4833-94e2-598b2abe84a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:55.012401 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3c6215-5f91-4833-94e2-598b2abe84a1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:55.012794 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3c6215-5f91-4833-94e2-598b2abe84a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:55.014469 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3c6215-5f91-4833-94e2-598b2abe84a1 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:55.023616 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 324/647] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 15:28:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:55.029012 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3c6215-5f91-4833-94e2-598b2abe84a1 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:55.044330 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be3ce19-aca9-421a-94a2-6a87b9be8957 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:55.082224 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54de14b-c170-446c-b6f8-111ca37442dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:55.083176 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54de14b-c170-446c-b6f8-111ca37442dc None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:55.083698 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54de14b-c170-446c-b6f8-111ca37442dc None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:55.084183 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54de14b-c170-446c-b6f8-111ca37442dc None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:55.085003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54de14b-c170-446c-b6f8-111ca37442dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:55.085563 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54de14b-c170-446c-b6f8-111ca37442dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:55.087481 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54de14b-c170-446c-b6f8-111ca37442dc None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:55.105767 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 324/648] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:28:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:55.107117 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54de14b-c170-446c-b6f8-111ca37442dc None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:55.128704 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3c6215-5f91-4833-94e2-598b2abe84a1 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:55.134244 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:55.163467 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5754e841-f82b-4161-8467-13c4a98226c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:55.164810 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5754e841-f82b-4161-8467-13c4a98226c8 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:55.164942 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5754e841-f82b-4161-8467-13c4a98226c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:55.165232 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5754e841-f82b-4161-8467-13c4a98226c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:55.176086 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5754e841-f82b-4161-8467-13c4a98226c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:55.177702 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5754e841-f82b-4161-8467-13c4a98226c8 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:55.192536 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-5754e841-f82b-4161-8467-13c4a98226c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:55.192536 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:55.192536 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:55.192536 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:55.192536 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:55.192536 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:55.192536 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:55.192536 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:55.192536 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:55.192536 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:55.192536 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:55.192536 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:55.192536 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:28:55.192536 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:28:55.192536 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:28:55.192536 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:28:55.193541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:55.193541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:55.193541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:55.193541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:55.193541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:55.193541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:55.193541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:55.193541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:55.193541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:55.193541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:55.193541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:55.193541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:55.193541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:55.193541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:55.193541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:55.193541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:55.193541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:28:55.194458 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:28:55.194458 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:55.194458 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:55.194458 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 325/649] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:55.196122 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5754e841-f82b-4161-8467-13c4a98226c8 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:55.205292 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:55.224662 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54de14b-c170-446c-b6f8-111ca37442dc None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:55.266961 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d5efa9-33e9-40f7-8e72-7631d3b156bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:55.267919 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d5efa9-33e9-40f7-8e72-7631d3b156bf None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:55.268378 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d5efa9-33e9-40f7-8e72-7631d3b156bf None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:55.268892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d5efa9-33e9-40f7-8e72-7631d3b156bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:55.280756 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d5efa9-33e9-40f7-8e72-7631d3b156bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:55.282694 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d5efa9-33e9-40f7-8e72-7631d3b156bf None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:55.300497 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-c4d5efa9-33e9-40f7-8e72-7631d3b156bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:55.300497 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:55.300497 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:55.300497 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:55.300497 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:55.300497 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:55.300497 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:55.300497 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:55.300497 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:55.300497 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:55.300497 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:55.300497 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:55.300497 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:28:55.300497 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:28:55.300497 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:28:55.300497 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:28:55.301445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:55.301445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:55.301445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:55.301445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:55.301445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:55.301445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:55.301445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:55.301445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:55.301445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:55.301445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:55.301445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:55.301445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:55.301445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:55.301445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:55.301445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:55.301445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:55.301445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:28:55.302405 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:28:55.302405 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:55.302405 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:55.309238 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 325/650] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:55.309426 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5754e841-f82b-4161-8467-13c4a98226c8 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:55.310659 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d5efa9-33e9-40f7-8e72-7631d3b156bf None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:55.345945 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4b3ea3-7c3f-44a6-8644-4acbdd2af988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:55.346691 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b3ea3-7c3f-44a6-8644-4acbdd2af988 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:55.347143 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b3ea3-7c3f-44a6-8644-4acbdd2af988 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:55.347297 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4b3ea3-7c3f-44a6-8644-4acbdd2af988 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:55.355171 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4b3ea3-7c3f-44a6-8644-4acbdd2af988 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:55.357080 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4b3ea3-7c3f-44a6-8644-4acbdd2af988 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:55.368063 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-2f4b3ea3-7c3f-44a6-8644-4acbdd2af988 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 28 15:28:55.368063 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:55.368063 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:55.368063 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:55.368063 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:55.368063 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:55.368063 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:55.368063 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:55.368063 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:55.368063 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:55.368063 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:55.368063 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:55.368063 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:28:55.368063 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:28:55.368063 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:28:55.368063 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:28:55.369094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:55.369094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:55.369094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:28:55.369094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:28:55.369094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:28:55.369094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:28:55.369094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:55.369094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:55.369094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:55.369094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:55.369094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:55.369094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:55.369094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:55.369094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:55.369094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:55.369094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:55.369094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:55.369094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:55.370099 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:55.370099 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:55.370099 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:28:55.370099 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:28:55.370099 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:28:55.370099 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:28:55.370099 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:28:55.370099 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:28:55.370099 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 28 15:28:55.370099 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:55.374034 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 326/651] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 15:28:55 2022] GET /identity/v3/users/neutron => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:55.374779 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4b3ea3-7c3f-44a6-8644-4acbdd2af988 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:55.406220 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:55.426854 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d5efa9-33e9-40f7-8e72-7631d3b156bf None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:55.440837 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:55.466737 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771da68a-5450-4e1a-a597-12df4cb20be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:55.467871 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771da68a-5450-4e1a-a597-12df4cb20be5 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:55.468377 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771da68a-5450-4e1a-a597-12df4cb20be5 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:55.468852 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771da68a-5450-4e1a-a597-12df4cb20be5 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:55.469700 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771da68a-5450-4e1a-a597-12df4cb20be5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:55.470401 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771da68a-5450-4e1a-a597-12df4cb20be5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:55.472069 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771da68a-5450-4e1a-a597-12df4cb20be5 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:55.476397 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4b3ea3-7c3f-44a6-8644-4acbdd2af988 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:55.485933 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 326/652] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:28:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:55.487160 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771da68a-5450-4e1a-a597-12df4cb20be5 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:55.511920 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee5173-08a4-424a-aae2-51fd00599a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:55.512560 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee5173-08a4-424a-aae2-51fd00599a81 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:55.513187 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee5173-08a4-424a-aae2-51fd00599a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:55.513187 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee5173-08a4-424a-aae2-51fd00599a81 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:55.513705 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cee5173-08a4-424a-aae2-51fd00599a81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:55.514029 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cee5173-08a4-424a-aae2-51fd00599a81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:55.515396 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cee5173-08a4-424a-aae2-51fd00599a81 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:55.532054 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 327/653] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:28:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:55.532054 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee5173-08a4-424a-aae2-51fd00599a81 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:55.609420 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771da68a-5450-4e1a-a597-12df4cb20be5 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:55.630224 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee5173-08a4-424a-aae2-51fd00599a81 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:55.642222 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:55.654369 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0eda38-b550-45aa-be55-a4667699a48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:55.655690 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0eda38-b550-45aa-be55-a4667699a48c None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:55.656467 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0eda38-b550-45aa-be55-a4667699a48c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:55.656973 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0eda38-b550-45aa-be55-a4667699a48c None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:55.657660 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:55.659919 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0eda38-b550-45aa-be55-a4667699a48c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:55.660429 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0eda38-b550-45aa-be55-a4667699a48c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:55.663476 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0eda38-b550-45aa-be55-a4667699a48c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:55.667491 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a3e1a-911b-4836-8a9c-361cb1176f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:55.668116 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a3e1a-911b-4836-8a9c-361cb1176f2c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:55.669115 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a3e1a-911b-4836-8a9c-361cb1176f2c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:55.669115 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a3e1a-911b-4836-8a9c-361cb1176f2c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:55.676828 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a3e1a-911b-4836-8a9c-361cb1176f2c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:55.678712 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a3e1a-911b-4836-8a9c-361cb1176f2c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:55.690907 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 327/654] 10.222.0.52 () {56 vars in 1139 bytes} [Wed Sep 28 15:28:55 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:55.692407 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0eda38-b550-45aa-be55-a4667699a48c None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:55.696239 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-5d7a3e1a-911b-4836-8a9c-361cb1176f2c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 28 15:28:55.696239 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:55.696239 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:55.696239 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:55.696239 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:55.696239 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:55.696239 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:55.696239 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:55.696239 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:55.696239 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:55.696239 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:55.696239 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:55.696239 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:28:55.696239 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:28:55.696239 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:28:55.696239 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:28:55.697193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:55.697193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:55.697193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:28:55.697193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:28:55.697193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:28:55.697193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:28:55.697193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:55.697193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:55.697193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:55.697193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:55.697193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:55.697193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:55.697193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:55.697193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:55.697193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:55.697193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:55.697193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:55.697193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:55.698144 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:55.698144 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:55.698144 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:28:55.698144 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:28:55.698144 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:28:55.698144 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:28:55.698144 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:28:55.698144 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:28:55.698144 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 28 15:28:55.698144 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:55.698144 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 328/655] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:28:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:55.699254 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a3e1a-911b-4836-8a9c-361cb1176f2c None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:55.809451 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a3e1a-911b-4836-8a9c-361cb1176f2c None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:55.834665 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0eda38-b550-45aa-be55-a4667699a48c None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:55.844344 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5124c253-6dab-4b59-8b3e-09aafb4031bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:55.844996 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5124c253-6dab-4b59-8b3e-09aafb4031bb None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:55.845293 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5124c253-6dab-4b59-8b3e-09aafb4031bb None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:55.845544 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5124c253-6dab-4b59-8b3e-09aafb4031bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:55.880272 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37694a40-5089-43d2-a002-192f7546de65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:55.881500 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37694a40-5089-43d2-a002-192f7546de65 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:55.882115 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37694a40-5089-43d2-a002-192f7546de65 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:55.882846 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37694a40-5089-43d2-a002-192f7546de65 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:55.893793 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37694a40-5089-43d2-a002-192f7546de65 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:55.896766 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37694a40-5089-43d2-a002-192f7546de65 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:55.912064 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-37694a40-5089-43d2-a002-192f7546de65 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 28 15:28:55.912064 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:55.912064 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:55.912064 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:55.912064 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:55.912064 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:55.912064 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:55.912064 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:55.912064 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:55.912064 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:55.912064 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:55.912064 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:55.912064 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:28:55.912064 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:28:55.912064 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:28:55.912064 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:28:55.913015 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:55.913015 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:55.913015 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:28:55.913015 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:28:55.913015 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:28:55.913015 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:28:55.913015 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:55.913015 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:55.913015 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:55.913015 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:55.913015 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:55.913015 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:55.913015 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:55.913015 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:55.913015 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:55.913015 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:55.913015 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:55.913015 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:55.913968 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:55.913968 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:55.913968 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:28:55.913968 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:28:55.913968 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:28:55.913968 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:28:55.913968 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:28:55.913968 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:28:55.913968 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 28 15:28:55.913968 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:55.914544 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:55.916566 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 328/656] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:28:55 2022] GET /identity/v3/users/glance => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:55.917700 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37694a40-5089-43d2-a002-192f7546de65 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:55.926828 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5124c253-6dab-4b59-8b3e-09aafb4031bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:55.928993 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5124c253-6dab-4b59-8b3e-09aafb4031bb None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:55.969838 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:56.026963 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37694a40-5089-43d2-a002-192f7546de65 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:56.037963 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 329/657] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:28:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:28:56.038883 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5124c253-6dab-4b59-8b3e-09aafb4031bb None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:56.063079 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a8fb00-b58d-4537-88e8-eaec36ad2c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:56.064035 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8fb00-b58d-4537-88e8-eaec36ad2c46 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:56.064507 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8fb00-b58d-4537-88e8-eaec36ad2c46 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:56.064991 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8fb00-b58d-4537-88e8-eaec36ad2c46 None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:56.065945 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a8fb00-b58d-4537-88e8-eaec36ad2c46 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:56.066445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a8fb00-b58d-4537-88e8-eaec36ad2c46 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:56.068064 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a8fb00-b58d-4537-88e8-eaec36ad2c46 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:56.087239 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 329/658] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Sep 28 15:28:55 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:56.088375 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a8fb00-b58d-4537-88e8-eaec36ad2c46 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:56.127361 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:56.143995 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5124c253-6dab-4b59-8b3e-09aafb4031bb None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:56.180055 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2ba1c9-b3ce-46df-85e8-a2c95905ad54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:56.181257 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ba1c9-b3ce-46df-85e8-a2c95905ad54 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:56.181257 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ba1c9-b3ce-46df-85e8-a2c95905ad54 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:56.181668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ba1c9-b3ce-46df-85e8-a2c95905ad54 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:56.182469 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2ba1c9-b3ce-46df-85e8-a2c95905ad54 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:56.183054 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2ba1c9-b3ce-46df-85e8-a2c95905ad54 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:56.184256 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2ba1c9-b3ce-46df-85e8-a2c95905ad54 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:56.196683 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a8fb00-b58d-4537-88e8-eaec36ad2c46 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:56.204269 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 330/659] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Sep 28 15:28:56 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:56.205258 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2ba1c9-b3ce-46df-85e8-a2c95905ad54 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:56.207560 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:56.243406 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6df09d-8300-4d2e-90d0-3bf38b380f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:56.244024 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6df09d-8300-4d2e-90d0-3bf38b380f71 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:56.244322 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6df09d-8300-4d2e-90d0-3bf38b380f71 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:56.244552 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6df09d-8300-4d2e-90d0-3bf38b380f71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:56.263162 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6df09d-8300-4d2e-90d0-3bf38b380f71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:56.264981 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6df09d-8300-4d2e-90d0-3bf38b380f71 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:56.289519 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-8f6df09d-8300-4d2e-90d0-3bf38b380f71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:56.289519 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:56.289519 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:56.289519 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:56.289519 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:56.289519 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:56.289519 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:56.289519 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:56.289519 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:56.289519 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:56.289519 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:56.289519 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:56.289519 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:28:56.289519 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:28:56.289519 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:28:56.289519 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:28:56.290551 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:56.290551 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:56.290551 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:56.290551 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:56.290551 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:56.290551 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:56.290551 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:56.290551 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:56.290551 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:56.290551 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:56.290551 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:56.290551 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:56.290551 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:56.290551 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:56.290551 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:56.290551 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:56.290551 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:28:56.291429 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:28:56.291429 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:56.291429 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:56.294340 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 330/660] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:56.295400 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6df09d-8300-4d2e-90d0-3bf38b380f71 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:56.333260 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2ba1c9-b3ce-46df-85e8-a2c95905ad54 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:56.379814 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a7abb9-618e-4490-b388-6a95b1e6107a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:56.380579 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a7abb9-618e-4490-b388-6a95b1e6107a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:56.381743 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a7abb9-618e-4490-b388-6a95b1e6107a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:56.382135 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a7abb9-618e-4490-b388-6a95b1e6107a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:56.400808 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:56.421113 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6df09d-8300-4d2e-90d0-3bf38b380f71 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:56.457462 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a7abb9-618e-4490-b388-6a95b1e6107a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:56.459165 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a7abb9-618e-4490-b388-6a95b1e6107a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:56.465371 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f765b3e2-2bc2-4128-bd77-806c15221416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:56.466146 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765b3e2-2bc2-4128-bd77-806c15221416 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:56.466520 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765b3e2-2bc2-4128-bd77-806c15221416 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:56.466933 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765b3e2-2bc2-4128-bd77-806c15221416 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:56.500012 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:56.549541 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f765b3e2-2bc2-4128-bd77-806c15221416 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:56.551379 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f765b3e2-2bc2-4128-bd77-806c15221416 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:56.594297 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 331/661] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:28:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:28:56.595430 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a7abb9-618e-4490-b388-6a95b1e6107a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:56.655879 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:56.662328 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 331/662] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:28:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:28:56.662979 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f765b3e2-2bc2-4128-bd77-806c15221416 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:56.721988 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:56.725723 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a7abb9-618e-4490-b388-6a95b1e6107a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:56.765721 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f765b3e2-2bc2-4128-bd77-806c15221416 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:56.770540 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622a57d7-c726-4a18-8b56-f11cbddd61c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:56.771138 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a57d7-c726-4a18-8b56-f11cbddd61c7 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:56.771422 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a57d7-c726-4a18-8b56-f11cbddd61c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:56.771692 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a57d7-c726-4a18-8b56-f11cbddd61c7 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:56.772314 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622a57d7-c726-4a18-8b56-f11cbddd61c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:56.772608 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622a57d7-c726-4a18-8b56-f11cbddd61c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:56.774027 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622a57d7-c726-4a18-8b56-f11cbddd61c7 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:56.790973 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 332/663] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:28:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:56.790973 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3682fb-6db0-4db5-8e43-93ef10f1b261 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:56.790973 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3682fb-6db0-4db5-8e43-93ef10f1b261 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:56.790973 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3682fb-6db0-4db5-8e43-93ef10f1b261 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:56.791339 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 333/664] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:56.791950 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3682fb-6db0-4db5-8e43-93ef10f1b261 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:56.804119 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e6c166-1921-4637-83c3-4b41cb47e006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:56.804782 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e6c166-1921-4637-83c3-4b41cb47e006 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:56.805553 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e6c166-1921-4637-83c3-4b41cb47e006 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:56.805830 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e6c166-1921-4637-83c3-4b41cb47e006 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:56.816139 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e6c166-1921-4637-83c3-4b41cb47e006 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:56.817657 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e6c166-1921-4637-83c3-4b41cb47e006 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:56.832971 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-f0e6c166-1921-4637-83c3-4b41cb47e006 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:56.832971 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:56.832971 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:56.832971 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:56.832971 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:56.832971 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:56.832971 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:56.832971 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:56.832971 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:56.832971 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:56.832971 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:56.832971 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:56.832971 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:28:56.832971 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:28:56.832971 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:28:56.832971 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:28:56.834231 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:56.834231 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:56.834231 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:56.834231 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:56.834231 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:56.834231 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:56.834231 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:56.834231 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:56.834231 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:56.834231 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:56.834231 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:56.834231 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:56.834231 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:56.834231 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:56.834231 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:56.834231 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:56.834231 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:28:56.835780 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:28:56.835780 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:56.835780 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:56.838695 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 332/665] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:56.839298 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e6c166-1921-4637-83c3-4b41cb47e006 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:56.902837 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3682fb-6db0-4db5-8e43-93ef10f1b261 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:56.906309 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:56.943808 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1672ae2a-18b5-4589-89e8-f3c6e5dd87bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:56.944596 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1672ae2a-18b5-4589-89e8-f3c6e5dd87bd None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:56.944877 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1672ae2a-18b5-4589-89e8-f3c6e5dd87bd None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:56.945149 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1672ae2a-18b5-4589-89e8-f3c6e5dd87bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:56.952863 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e6c166-1921-4637-83c3-4b41cb47e006 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:56.956457 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1672ae2a-18b5-4589-89e8-f3c6e5dd87bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:56.957931 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1672ae2a-18b5-4589-89e8-f3c6e5dd87bd None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:56.963830 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:56.974435 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-1672ae2a-18b5-4589-89e8-f3c6e5dd87bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:56.974435 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:56.974435 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:56.974435 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:56.974435 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:56.974435 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:56.974435 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:56.974435 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:56.974435 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:56.974435 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:56.974435 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:56.974435 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:56.974435 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:28:56.974435 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:28:56.974435 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:28:56.974435 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:28:56.975394 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:56.975394 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:56.975394 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:56.975394 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:56.975394 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:56.975394 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:56.975394 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:56.975394 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:56.975394 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:56.975394 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:56.975394 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:56.975394 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:56.975394 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:56.975394 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:56.975394 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:56.975394 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:56.975394 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:28:56.976292 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:28:56.976292 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:56.976292 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:56.976292 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 334/666] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:56.977327 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b08e12-615d-4436-bb50-93ec59226868 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:56.977465 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b08e12-615d-4436-bb50-93ec59226868 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:56.977724 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b08e12-615d-4436-bb50-93ec59226868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:56.994341 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c53c31-afb2-40a3-82a2-334e8cc9399a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:56.995272 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c53c31-afb2-40a3-82a2-334e8cc9399a None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:56.995732 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c53c31-afb2-40a3-82a2-334e8cc9399a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:56.996197 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c53c31-afb2-40a3-82a2-334e8cc9399a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:56.998070 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-15b08e12-615d-4436-bb50-93ec59226868 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:56.999232 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-15b08e12-615d-4436-bb50-93ec59226868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:57.003559 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c53c31-afb2-40a3-82a2-334e8cc9399a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:57.005245 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c53c31-afb2-40a3-82a2-334e8cc9399a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:57.017110 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-78c53c31-afb2-40a3-82a2-334e8cc9399a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:28:57.017110 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:57.017110 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:57.017110 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:57.017110 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:57.017110 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:57.017110 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:57.017110 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:57.017110 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:57.017110 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:57.017110 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:57.017110 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:57.017110 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:28:57.017110 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:28:57.017110 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:28:57.018057 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:28:57.018057 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:57.018057 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:57.018057 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:57.018057 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:57.018057 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:57.018057 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:57.018057 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:57.018057 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:57.018057 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:57.018057 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:57.018057 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:57.018057 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:57.018057 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:57.018057 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:57.018057 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:57.018057 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:57.018965 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:28:57.018965 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:28:57.018965 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:28:57.018965 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:28:57.018965 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:28:57.018965 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:28:57.018965 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:28:57.018965 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:57.018965 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 333/667] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:28:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:57.018965 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c53c31-afb2-40a3-82a2-334e8cc9399a None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:57.065594 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 335/668] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:57.066446 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b08e12-615d-4436-bb50-93ec59226868 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:57.122664 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c53c31-afb2-40a3-82a2-334e8cc9399a None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:57.150318 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:57.158272 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a471f94b-8afc-4526-9f11-e3bc44dc0ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:57.158872 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a471f94b-8afc-4526-9f11-e3bc44dc0ae6 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:57.159142 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a471f94b-8afc-4526-9f11-e3bc44dc0ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:57.159405 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a471f94b-8afc-4526-9f11-e3bc44dc0ae6 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:57.160036 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a471f94b-8afc-4526-9f11-e3bc44dc0ae6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:57.160333 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a471f94b-8afc-4526-9f11-e3bc44dc0ae6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:57.161671 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a471f94b-8afc-4526-9f11-e3bc44dc0ae6 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:57.166729 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b08e12-615d-4436-bb50-93ec59226868 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:57.178893 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 334/669] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:28:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:57.179978 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c68ebd-fdf0-4e05-8c25-65e20c2b706f None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:57.180259 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c68ebd-fdf0-4e05-8c25-65e20c2b706f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:57.180535 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c68ebd-fdf0-4e05-8c25-65e20c2b706f None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:57.181993 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 335/670] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:28:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:28:57.182680 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c68ebd-fdf0-4e05-8c25-65e20c2b706f None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:57.204965 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5622f2-7fcb-4cdb-b34a-b9caee2df82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:57.205574 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5622f2-7fcb-4cdb-b34a-b9caee2df82c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:57.205861 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5622f2-7fcb-4cdb-b34a-b9caee2df82c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:57.206161 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5622f2-7fcb-4cdb-b34a-b9caee2df82c None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:57.206794 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5622f2-7fcb-4cdb-b34a-b9caee2df82c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:57.207086 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5622f2-7fcb-4cdb-b34a-b9caee2df82c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:57.208446 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5622f2-7fcb-4cdb-b34a-b9caee2df82c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:57.213976 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:57.228753 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 336/671] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:28:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:57.228753 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5622f2-7fcb-4cdb-b34a-b9caee2df82c None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:57.300257 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c68ebd-fdf0-4e05-8c25-65e20c2b706f None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:57.337576 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68a7836-d34a-4491-ad71-5824aba4671c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:57.338329 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a7836-d34a-4491-ad71-5824aba4671c None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:57.338602 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a7836-d34a-4491-ad71-5824aba4671c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:57.338867 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a7836-d34a-4491-ad71-5824aba4671c None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:57.339541 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68a7836-d34a-4491-ad71-5824aba4671c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:57.339839 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68a7836-d34a-4491-ad71-5824aba4671c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:57.340291 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5622f2-7fcb-4cdb-b34a-b9caee2df82c None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:57.341286 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68a7836-d34a-4491-ad71-5824aba4671c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:57.352731 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 336/672] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 15:28:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:57.354094 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ce3c2e-bb09-4404-967e-ac820f8d134e None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:57.354385 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ce3c2e-bb09-4404-967e-ac820f8d134e None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:57.354656 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ce3c2e-bb09-4404-967e-ac820f8d134e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:57.372067 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-44ce3c2e-bb09-4404-967e-ac820f8d134e None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:28:57.373229 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-44ce3c2e-bb09-4404-967e-ac820f8d134e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:28:57.374650 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd5a62-b49d-42e3-b101-e8d8668faa97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:57.375272 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd5a62-b49d-42e3-b101-e8d8668faa97 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:57.375549 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd5a62-b49d-42e3-b101-e8d8668faa97 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:57.375820 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd5a62-b49d-42e3-b101-e8d8668faa97 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:57.382440 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dd5a62-b49d-42e3-b101-e8d8668faa97 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:57.384525 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dd5a62-b49d-42e3-b101-e8d8668faa97 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:57.395283 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-22dd5a62-b49d-42e3-b101-e8d8668faa97 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:28:57.395283 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:57.395283 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:57.395283 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:57.395283 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:57.395283 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:57.395283 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:57.395283 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:57.395283 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:57.395283 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:57.395283 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:57.395283 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:57.395283 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:28:57.395283 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:28:57.395283 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:28:57.396194 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:28:57.396194 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:57.396194 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:57.396194 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:57.396194 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:57.396194 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:57.396194 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:57.396194 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:57.396194 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:57.396194 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:57.396194 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:57.396194 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:57.396194 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:57.396194 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:57.396194 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:57.396194 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:57.396194 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:57.397100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:28:57.397100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:28:57.397100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:28:57.397100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:28:57.397100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:28:57.397100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:28:57.397100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:28:57.397100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:57.397100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 337/673] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:28:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:57.397100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd5a62-b49d-42e3-b101-e8d8668faa97 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:57.405956 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:57.443389 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 337/674] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:28:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:28:57.444422 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ce3c2e-bb09-4404-967e-ac820f8d134e None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:57.472228 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:57.503017 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd5a62-b49d-42e3-b101-e8d8668faa97 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:57.537396 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bcc97d-2969-4c5e-92cc-caf4d92052e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:57.538155 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bcc97d-2969-4c5e-92cc-caf4d92052e0 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:57.538433 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bcc97d-2969-4c5e-92cc-caf4d92052e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:57.538698 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bcc97d-2969-4c5e-92cc-caf4d92052e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:57.546486 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bcc97d-2969-4c5e-92cc-caf4d92052e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:57.548136 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bcc97d-2969-4c5e-92cc-caf4d92052e0 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:57.553528 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ce3c2e-bb09-4404-967e-ac820f8d134e None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:57.560145 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-99bcc97d-2969-4c5e-92cc-caf4d92052e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:28:57.560145 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:57.560145 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:57.560145 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:57.560145 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:57.560145 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:57.560145 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:57.560145 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:57.560145 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:57.560145 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:57.560145 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:57.560145 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:57.560145 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:28:57.560145 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:28:57.560145 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:28:57.561046 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:28:57.561046 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:57.561046 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:57.561046 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:57.561046 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:57.561046 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:57.561046 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:57.561046 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:57.561046 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:57.561046 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:57.561046 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:57.561046 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:57.561046 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:57.561046 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:57.561046 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:57.561046 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:57.561046 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:57.561966 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:28:57.561966 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:28:57.561966 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:28:57.561966 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:28:57.561966 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:28:57.561966 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:28:57.561966 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:28:57.561966 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:57.561966 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 338/675] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:28:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:57.563004 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bcc97d-2969-4c5e-92cc-caf4d92052e0 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:57.596118 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091f8f6d-fbce-41a2-ba6f-ab4eb8bf960d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:57.596745 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f8f6d-fbce-41a2-ba6f-ab4eb8bf960d None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:57.597022 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f8f6d-fbce-41a2-ba6f-ab4eb8bf960d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:57.597290 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f8f6d-fbce-41a2-ba6f-ab4eb8bf960d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:57.609667 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091f8f6d-fbce-41a2-ba6f-ab4eb8bf960d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:57.614526 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-091f8f6d-fbce-41a2-ba6f-ab4eb8bf960d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:28:57.614526 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:57.614526 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:57.614526 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:57.614526 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:57.614526 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:57.614526 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:57.614526 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:57.614526 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:57.614526 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:57.614526 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:57.614526 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:57.614526 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:57.614526 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:57.614526 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:57.614526 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:57.617897 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:57.617897 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:57.617897 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:57.617897 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:57.617897 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:57.617897 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:57.617897 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:57.617897 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:57.617897 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:57.617897 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:57.617897 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:57.617897 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:57.617897 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:57.617897 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:57.617897 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:57.617897 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:57.618824 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:57.618824 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:57.618824 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:57.618824 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:57.618824 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:57.618824 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:57.618824 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:57.618824 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:57.618824 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:57.618824 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:57.618824 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:28:57.618824 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:57.618824 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 338/676] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 15:28:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:57.618824 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091f8f6d-fbce-41a2-ba6f-ab4eb8bf960d None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:57.644451 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:57.683388 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bcc97d-2969-4c5e-92cc-caf4d92052e0 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:57.702681 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:57.718961 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fd8dad-3380-41bf-a159-c5e98a98264c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:57.719562 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd8dad-3380-41bf-a159-c5e98a98264c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:57.719838 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd8dad-3380-41bf-a159-c5e98a98264c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:57.720102 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd8dad-3380-41bf-a159-c5e98a98264c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:57.720836 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fd8dad-3380-41bf-a159-c5e98a98264c None admin] RBAC: Adding query filter params (role.id=342b64fe0cdb4925b23c6cd461a152ac, scope.project.id=3dc6c5f5fcd34922b0d09cd2aaed6cf1, user.id=b13b0533444a449ead00a389c5f6bd1a) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:57.721135 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fd8dad-3380-41bf-a159-c5e98a98264c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:57.722581 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fd8dad-3380-41bf-a159-c5e98a98264c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:57.723115 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091f8f6d-fbce-41a2-ba6f-ab4eb8bf960d None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:57.728097 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 339/677] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 15:28:57 2022] GET /identity/v3/role_assignments?user.id=b13b0533444a449ead00a389c5f6bd1a&scope.project.id=3dc6c5f5fcd34922b0d09cd2aaed6cf1&role.id=342b64fe0cdb4925b23c6cd461a152ac => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:57.728857 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fd8dad-3380-41bf-a159-c5e98a98264c None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:57.773470 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3511e5c6-9ac5-4953-932e-f7f17b4052dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:57.774183 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3511e5c6-9ac5-4953-932e-f7f17b4052dc None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:57.774464 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3511e5c6-9ac5-4953-932e-f7f17b4052dc None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:57.774722 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3511e5c6-9ac5-4953-932e-f7f17b4052dc None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:57.775407 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3511e5c6-9ac5-4953-932e-f7f17b4052dc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:57.775704 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3511e5c6-9ac5-4953-932e-f7f17b4052dc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:57.777256 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3511e5c6-9ac5-4953-932e-f7f17b4052dc None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:57.796964 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 339/678] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 15:28:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:57.797950 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3511e5c6-9ac5-4953-932e-f7f17b4052dc None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:57.838277 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fd8dad-3380-41bf-a159-c5e98a98264c None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:57.869730 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:57.876508 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ddf90c-b740-4d94-bc2a-2e4c2bd37cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:57.877136 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ddf90c-b740-4d94-bc2a-2e4c2bd37cfa None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:57.877417 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ddf90c-b740-4d94-bc2a-2e4c2bd37cfa None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:57.877730 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ddf90c-b740-4d94-bc2a-2e4c2bd37cfa None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:57.878427 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ddf90c-b740-4d94-bc2a-2e4c2bd37cfa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:57.878735 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ddf90c-b740-4d94-bc2a-2e4c2bd37cfa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:57.880198 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ddf90c-b740-4d94-bc2a-2e4c2bd37cfa None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:57.899181 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 340/679] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 15:28:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:57.899181 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ddf90c-b740-4d94-bc2a-2e4c2bd37cfa None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:57.919247 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3511e5c6-9ac5-4953-932e-f7f17b4052dc None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:57.956716 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46dcc9b-e4fa-45cf-b7d8-81cf59e67cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:57.957322 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46dcc9b-e4fa-45cf-b7d8-81cf59e67cff None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:57.957596 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46dcc9b-e4fa-45cf-b7d8-81cf59e67cff None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:57.957879 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46dcc9b-e4fa-45cf-b7d8-81cf59e67cff None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:57.958537 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46dcc9b-e4fa-45cf-b7d8-81cf59e67cff None admin] RBAC: Adding query filter params (name=admin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:57.958837 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46dcc9b-e4fa-45cf-b7d8-81cf59e67cff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:57.960189 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46dcc9b-e4fa-45cf-b7d8-81cf59e67cff None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:57.971322 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 340/680] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:28:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:57.972713 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46dcc9b-e4fa-45cf-b7d8-81cf59e67cff None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:57.976082 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:58.013648 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ddf90c-b740-4d94-bc2a-2e4c2bd37cfa None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:58.050014 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69d161f-bce4-4295-a0ac-000d15714846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:58.050557 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d161f-bce4-4295-a0ac-000d15714846 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:58.050834 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d161f-bce4-4295-a0ac-000d15714846 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:58.051102 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d161f-bce4-4295-a0ac-000d15714846 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:58.061347 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69d161f-bce4-4295-a0ac-000d15714846 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:28:58.066107 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-b69d161f-bce4-4295-a0ac-000d15714846 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:28:58.066107 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:58.066107 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:58.066107 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:58.066107 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:58.066107 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:58.066107 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:58.066107 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:58.066107 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:58.066107 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:58.066107 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:58.066107 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:58.066107 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:28:58.066107 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:28:58.066107 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:28:58.066107 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:28:58.067032 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:28:58.067032 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:28:58.067032 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:28:58.067032 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:28:58.067032 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:58.067032 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:58.067032 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:58.067032 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:58.067032 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:58.067032 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:58.067032 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:58.067032 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:58.067032 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:58.067032 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:58.067032 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:58.067032 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:58.067856 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:58.067856 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:58.067856 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:58.067856 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:58.067856 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:28:58.067856 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:28:58.067856 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:28:58.067856 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:28:58.067856 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:28:58.067856 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:28:58.067856 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:28:58.067856 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:58.067856 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 341/681] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 15:28:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:58.068537 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69d161f-bce4-4295-a0ac-000d15714846 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:58.088865 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46dcc9b-e4fa-45cf-b7d8-81cf59e67cff None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:58.123747 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:58.124099 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7849a26e-2072-4a66-9fb2-a51a81af4430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:58.124710 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849a26e-2072-4a66-9fb2-a51a81af4430 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:58.124988 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849a26e-2072-4a66-9fb2-a51a81af4430 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:58.125267 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849a26e-2072-4a66-9fb2-a51a81af4430 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:58.126037 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7849a26e-2072-4a66-9fb2-a51a81af4430 None admin] RBAC: Adding query filter params (role.id=342b64fe0cdb4925b23c6cd461a152ac, scope.project.id=3dc6c5f5fcd34922b0d09cd2aaed6cf1, user.id=01833709177e4e68a2fa52fb9963d3bf) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:58.126369 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7849a26e-2072-4a66-9fb2-a51a81af4430 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:58.127798 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7849a26e-2072-4a66-9fb2-a51a81af4430 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:58.145895 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 341/682] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 15:28:57 2022] GET /identity/v3/role_assignments?user.id=01833709177e4e68a2fa52fb9963d3bf&scope.project.id=3dc6c5f5fcd34922b0d09cd2aaed6cf1&role.id=342b64fe0cdb4925b23c6cd461a152ac => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:58.145895 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7849a26e-2072-4a66-9fb2-a51a81af4430 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:58.194435 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69d161f-bce4-4295-a0ac-000d15714846 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:58.222567 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:58.236058 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9544dce-3008-4f4b-9317-d47e8bd598cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:58.236828 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9544dce-3008-4f4b-9317-d47e8bd598cf None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:58.237181 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9544dce-3008-4f4b-9317-d47e8bd598cf None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:58.237521 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9544dce-3008-4f4b-9317-d47e8bd598cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:58.238791 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9544dce-3008-4f4b-9317-d47e8bd598cf None admin] RBAC: Adding query filter params (role.id=342b64fe0cdb4925b23c6cd461a152ac, scope.project.id=3dc6c5f5fcd34922b0d09cd2aaed6cf1, user.id=c6a46bbd249c4434ad7d9442754f098a) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:58.239176 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9544dce-3008-4f4b-9317-d47e8bd598cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:58.240830 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9544dce-3008-4f4b-9317-d47e8bd598cf None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:58.248333 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 342/683] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 15:28:58 2022] GET /identity/v3/role_assignments?user.id=c6a46bbd249c4434ad7d9442754f098a&scope.project.id=3dc6c5f5fcd34922b0d09cd2aaed6cf1&role.id=342b64fe0cdb4925b23c6cd461a152ac => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:58.249191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9544dce-3008-4f4b-9317-d47e8bd598cf None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:58.307998 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7849a26e-2072-4a66-9fb2-a51a81af4430 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:58.379080 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9544dce-3008-4f4b-9317-d47e8bd598cf None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:58.391383 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9932f24a-4776-4d6e-9b28-c5fb677d36e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:58.392059 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932f24a-4776-4d6e-9b28-c5fb677d36e4 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:58.392339 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932f24a-4776-4d6e-9b28-c5fb677d36e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:58.393091 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932f24a-4776-4d6e-9b28-c5fb677d36e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:58.397185 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:58.410002 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9932f24a-4776-4d6e-9b28-c5fb677d36e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:58.412476 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9932f24a-4776-4d6e-9b28-c5fb677d36e4 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:58.423486 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7358f42-a126-4d54-9a66-4b9a0a499b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:58.424111 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7358f42-a126-4d54-9a66-4b9a0a499b09 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:58.424387 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7358f42-a126-4d54-9a66-4b9a0a499b09 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:58.424657 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7358f42-a126-4d54-9a66-4b9a0a499b09 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:58.425327 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7358f42-a126-4d54-9a66-4b9a0a499b09 None admin] RBAC: Adding query filter params (name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:58.425625 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7358f42-a126-4d54-9a66-4b9a0a499b09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:58.427084 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7358f42-a126-4d54-9a66-4b9a0a499b09 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:58.431409 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-9932f24a-4776-4d6e-9b28-c5fb677d36e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:58.431409 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:58.431409 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:58.431409 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:58.431409 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:58.431409 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:58.431409 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:58.431409 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:58.431409 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:58.431409 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:58.431409 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:58.431409 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:58.431409 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:28:58.431409 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:28:58.431409 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:28:58.431409 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:28:58.432414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:58.432414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:58.432414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:58.432414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:58.432414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:58.432414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:58.432414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:58.432414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:58.432414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:58.432414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:58.432414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:58.432414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:58.432414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:58.432414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:58.432414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:58.432414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:58.432414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:28:58.433298 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:28:58.433298 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:58.433298 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:58.433298 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 342/684] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:58.439023 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9932f24a-4776-4d6e-9b28-c5fb677d36e4 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:58.443256 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 343/685] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 15:28:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:58.449673 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7358f42-a126-4d54-9a66-4b9a0a499b09 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:58.558340 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9932f24a-4776-4d6e-9b28-c5fb677d36e4 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:58.567504 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7358f42-a126-4d54-9a66-4b9a0a499b09 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:58.582241 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:58.603568 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb2bfd-5ec3-42cc-84cc-3bf09a0f5ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:58.604230 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb2bfd-5ec3-42cc-84cc-3bf09a0f5ed4 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:58.604507 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb2bfd-5ec3-42cc-84cc-3bf09a0f5ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:58.604777 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb2bfd-5ec3-42cc-84cc-3bf09a0f5ed4 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:58.605409 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bb2bfd-5ec3-42cc-84cc-3bf09a0f5ed4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:58.605700 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bb2bfd-5ec3-42cc-84cc-3bf09a0f5ed4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:58.607100 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bb2bfd-5ec3-42cc-84cc-3bf09a0f5ed4 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:58.614572 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0718cc6-4a86-4225-9623-53905848b420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:58.616115 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0718cc6-4a86-4225-9623-53905848b420 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:58.616618 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0718cc6-4a86-4225-9623-53905848b420 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:58.617111 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0718cc6-4a86-4225-9623-53905848b420 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:58.621741 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 343/686] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:28:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:58.633167 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb2bfd-5ec3-42cc-84cc-3bf09a0f5ed4 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:58.637495 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0718cc6-4a86-4225-9623-53905848b420 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:58.639852 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0718cc6-4a86-4225-9623-53905848b420 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:58.660943 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-a0718cc6-4a86-4225-9623-53905848b420 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:58.660943 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:58.660943 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:58.660943 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:58.660943 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:58.660943 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:58.660943 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:58.660943 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:58.660943 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:58.660943 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:58.660943 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:58.660943 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:58.660943 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:28:58.660943 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:28:58.660943 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:28:58.660943 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:28:58.662184 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:58.662184 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:58.662184 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:58.662184 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:58.662184 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:58.662184 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:58.662184 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:58.662184 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:58.662184 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:58.662184 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:58.662184 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:58.662184 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:58.662184 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:58.662184 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:58.662184 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:58.662184 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:58.662184 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:28:58.663308 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:28:58.663308 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:58.663308 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:58.666332 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 344/687] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:28:58.674862 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0718cc6-4a86-4225-9623-53905848b420 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:58.744008 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:58.753165 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb2bfd-5ec3-42cc-84cc-3bf09a0f5ed4 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:58.791318 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ec25b-e731-4a1a-88f4-45bc2f6d6183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:58.792264 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ec25b-e731-4a1a-88f4-45bc2f6d6183 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:58.792640 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ec25b-e731-4a1a-88f4-45bc2f6d6183 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:58.793013 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ec25b-e731-4a1a-88f4-45bc2f6d6183 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:58.803857 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678ec25b-e731-4a1a-88f4-45bc2f6d6183 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:58.805805 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678ec25b-e731-4a1a-88f4-45bc2f6d6183 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:58.827319 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-678ec25b-e731-4a1a-88f4-45bc2f6d6183 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 28 15:28:58.827319 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:58.827319 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:58.827319 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:58.827319 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:58.827319 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:58.827319 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:58.827319 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:58.827319 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:58.827319 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:58.827319 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:58.827319 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:58.827319 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:28:58.827319 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:28:58.827319 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:28:58.827319 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:28:58.828283 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:58.828283 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:58.828283 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:28:58.828283 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:28:58.828283 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:28:58.828283 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:28:58.828283 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:58.828283 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:58.828283 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:58.828283 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:58.828283 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:58.828283 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:58.828283 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:58.828283 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:58.828283 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:58.828283 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:58.828283 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:58.828283 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:58.829247 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:58.829247 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:58.829247 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:28:58.829247 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:28:58.829247 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:28:58.829247 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:28:58.829247 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:28:58.829247 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:28:58.829247 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 28 15:28:58.829247 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:58.836218 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 344/688] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Sep 28 15:28:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:58.841292 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ec25b-e731-4a1a-88f4-45bc2f6d6183 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:58.861090 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0718cc6-4a86-4225-9623-53905848b420 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:58.900200 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afeadd0-87fb-4f2c-9fc2-fa2cc6b59852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:58.900853 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afeadd0-87fb-4f2c-9fc2-fa2cc6b59852 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:58.901129 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afeadd0-87fb-4f2c-9fc2-fa2cc6b59852 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:58.901450 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afeadd0-87fb-4f2c-9fc2-fa2cc6b59852 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:58.902146 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afeadd0-87fb-4f2c-9fc2-fa2cc6b59852 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:58.902445 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afeadd0-87fb-4f2c-9fc2-fa2cc6b59852 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:58.903801 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afeadd0-87fb-4f2c-9fc2-fa2cc6b59852 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:58.917879 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 345/689] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:28:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:58.923807 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afeadd0-87fb-4f2c-9fc2-fa2cc6b59852 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:58.924270 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:58.994384 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ec25b-e731-4a1a-88f4-45bc2f6d6183 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:59.031533 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f113b8-6314-418b-9995-ae4c0409e59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:59.032149 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f113b8-6314-418b-9995-ae4c0409e59a None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:59.032428 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f113b8-6314-418b-9995-ae4c0409e59a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:59.032702 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f113b8-6314-418b-9995-ae4c0409e59a None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:59.033428 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f113b8-6314-418b-9995-ae4c0409e59a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:59.033734 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f113b8-6314-418b-9995-ae4c0409e59a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:59.036770 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f113b8-6314-418b-9995-ae4c0409e59a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:59.053101 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 345/690] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Sep 28 15:28:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:59.058593 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f113b8-6314-418b-9995-ae4c0409e59a None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:59.106293 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:59.128842 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afeadd0-87fb-4f2c-9fc2-fa2cc6b59852 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:59.158522 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:59.164350 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f113b8-6314-418b-9995-ae4c0409e59a None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:59.196449 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6045b255-3632-4027-82fb-f90f98acf7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:59.197085 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6045b255-3632-4027-82fb-f90f98acf7bc None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:59.197362 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6045b255-3632-4027-82fb-f90f98acf7bc None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:59.197630 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6045b255-3632-4027-82fb-f90f98acf7bc None admin] PATH_INFO: `/v3/users/swift` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:59.200668 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6aba68-bab9-44c3-b568-ceb58bc0df98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:59.201366 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6aba68-bab9-44c3-b568-ceb58bc0df98 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:59.201716 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6aba68-bab9-44c3-b568-ceb58bc0df98 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:59.202085 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6aba68-bab9-44c3-b568-ceb58bc0df98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:59.205219 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6045b255-3632-4027-82fb-f90f98acf7bc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:59.206711 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6045b255-3632-4027-82fb-f90f98acf7bc None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:59.219168 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-6045b255-3632-4027-82fb-f90f98acf7bc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 28 15:28:59.219168 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:59.219168 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:59.219168 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:59.219168 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:59.219168 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:59.219168 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:59.219168 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:59.219168 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:59.219168 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:59.219168 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:59.219168 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:59.219168 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:28:59.219168 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:28:59.219168 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:28:59.219168 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:28:59.220128 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:59.220128 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:59.220128 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:28:59.220128 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:28:59.220128 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:28:59.220128 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:28:59.220128 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:59.220128 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:59.220128 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:59.220128 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:59.220128 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:59.220128 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:59.220128 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:59.220128 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:59.220128 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:59.220128 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:59.220128 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:59.220128 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:59.221076 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:59.221076 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:59.221076 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:28:59.221076 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:28:59.221076 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:28:59.221076 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:28:59.221076 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:28:59.221076 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:28:59.221076 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 28 15:28:59.221076 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:28:59.224893 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 346/691] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 15:28:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:28:59.226103 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6045b255-3632-4027-82fb-f90f98acf7bc None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:59.272932 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6aba68-bab9-44c3-b568-ceb58bc0df98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:59.274458 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6aba68-bab9-44c3-b568-ceb58bc0df98 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:59.326262 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6045b255-3632-4027-82fb-f90f98acf7bc None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:59.363626 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcacf9d3-4f1f-43a4-8320-9bd4cdeb3b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:59.364149 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcacf9d3-4f1f-43a4-8320-9bd4cdeb3b53 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:59.364424 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcacf9d3-4f1f-43a4-8320-9bd4cdeb3b53 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:59.364689 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcacf9d3-4f1f-43a4-8320-9bd4cdeb3b53 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:59.365396 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcacf9d3-4f1f-43a4-8320-9bd4cdeb3b53 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:59.365694 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcacf9d3-4f1f-43a4-8320-9bd4cdeb3b53 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:59.367835 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcacf9d3-4f1f-43a4-8320-9bd4cdeb3b53 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:59.407754 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 347/692] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Sep 28 15:28:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:28:59.409125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcacf9d3-4f1f-43a4-8320-9bd4cdeb3b53 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:59.430521 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:59.463719 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:59.480683 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 346/693] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:28:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 28 15:28:59.485938 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6aba68-bab9-44c3-b568-ceb58bc0df98 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:59.567197 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcacf9d3-4f1f-43a4-8320-9bd4cdeb3b53 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:59.605841 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b696b-76a2-4e1f-a73c-7467c9830ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:59.606464 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b696b-76a2-4e1f-a73c-7467c9830ad6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:59.606738 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b696b-76a2-4e1f-a73c-7467c9830ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:59.607004 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b696b-76a2-4e1f-a73c-7467c9830ad6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:59.677465 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6aba68-bab9-44c3-b568-ceb58bc0df98 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:59.719581 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320f176b-8174-4cbe-912c-295b839cb9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:59.720203 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320f176b-8174-4cbe-912c-295b839cb9a2 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:59.720476 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320f176b-8174-4cbe-912c-295b839cb9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:59.720743 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320f176b-8174-4cbe-912c-295b839cb9a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:59.743513 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320f176b-8174-4cbe-912c-295b839cb9a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:59.744950 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320f176b-8174-4cbe-912c-295b839cb9a2 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:59.746255 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21b696b-76a2-4e1f-a73c-7467c9830ad6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:59.747890 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21b696b-76a2-4e1f-a73c-7467c9830ad6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:59.770162 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:59.773068 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-320f176b-8174-4cbe-912c-295b839cb9a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:59.773068 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:28:59.773068 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:28:59.773068 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:28:59.773068 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:28:59.773068 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:28:59.773068 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:28:59.773068 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:28:59.773068 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:28:59.773068 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:28:59.773068 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:28:59.773068 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:28:59.773068 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:28:59.773068 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:28:59.773068 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:28:59.773068 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:28:59.774941 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:28:59.774941 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:28:59.774941 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:28:59.774941 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:28:59.774941 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:28:59.774941 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:28:59.774941 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:28:59.774941 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:28:59.774941 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:28:59.774941 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:28:59.774941 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:28:59.774941 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:28:59.774941 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:28:59.774941 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:28:59.774941 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:28:59.774941 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:28:59.774941 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:28:59.775814 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:28:59.775814 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:28:59.775814 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:28:59.780740 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 347/694] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:28:59.783035 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320f176b-8174-4cbe-912c-295b839cb9a2 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:59.844375 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:28:59.897251 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320f176b-8174-4cbe-912c-295b839cb9a2 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:28:59.913346 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 348/695] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:28:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 28 15:28:59.921110 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b696b-76a2-4e1f-a73c-7467c9830ad6 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:28:59.940123 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c116c4d2-344d-43ce-9a4c-8f88c64f30d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:28:59.941023 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c116c4d2-344d-43ce-9a4c-8f88c64f30d0 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:28:59.941498 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c116c4d2-344d-43ce-9a4c-8f88c64f30d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:28:59.942020 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c116c4d2-344d-43ce-9a4c-8f88c64f30d0 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:28:59.942991 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c116c4d2-344d-43ce-9a4c-8f88c64f30d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:28:59.943488 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c116c4d2-344d-43ce-9a4c-8f88c64f30d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:28:59.945327 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c116c4d2-344d-43ce-9a4c-8f88c64f30d0 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:28:59.959783 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 348/696] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:28:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:28:59.965145 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c116c4d2-344d-43ce-9a4c-8f88c64f30d0 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:00.099491 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b696b-76a2-4e1f-a73c-7467c9830ad6 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:00.108425 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:00.139783 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ca2c00-c8f9-4dbb-991d-772d4ee05103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:00.140829 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca2c00-c8f9-4dbb-991d-772d4ee05103 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:00.141369 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca2c00-c8f9-4dbb-991d-772d4ee05103 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:00.141880 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca2c00-c8f9-4dbb-991d-772d4ee05103 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:00.153352 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ca2c00-c8f9-4dbb-991d-772d4ee05103 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:00.155083 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ca2c00-c8f9-4dbb-991d-772d4ee05103 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:00.171206 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-18ca2c00-c8f9-4dbb-991d-772d4ee05103 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:00.171206 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:00.171206 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:00.171206 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:00.171206 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:00.171206 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:00.171206 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:00.171206 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:00.171206 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:00.171206 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:00.171206 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:00.171206 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:00.171206 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:00.171206 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:00.171206 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:00.171206 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:00.172220 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:00.172220 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:00.172220 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:00.172220 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:00.172220 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:00.172220 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:00.172220 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:00.172220 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:00.172220 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:00.172220 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:00.172220 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:00.172220 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:00.172220 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:00.172220 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:00.172220 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:00.172220 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:00.172220 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:00.173107 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:00.173107 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:00.173107 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:00.175253 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 349/697] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:28:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:29:00.181996 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ca2c00-c8f9-4dbb-991d-772d4ee05103 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:00.196021 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c116c4d2-344d-43ce-9a4c-8f88c64f30d0 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:00.205184 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:00.231305 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b6ce0-5987-4c9c-9150-b5f0cfa80ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:00.231930 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b6ce0-5987-4c9c-9150-b5f0cfa80ea0 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:00.232203 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b6ce0-5987-4c9c-9150-b5f0cfa80ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:00.232468 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b6ce0-5987-4c9c-9150-b5f0cfa80ea0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:00.239084 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8b6ce0-5987-4c9c-9150-b5f0cfa80ea0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:00.240616 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8b6ce0-5987-4c9c-9150-b5f0cfa80ea0 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:00.251872 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-bf8b6ce0-5987-4c9c-9150-b5f0cfa80ea0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:00.251872 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:00.251872 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:00.251872 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:00.251872 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:00.251872 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:00.251872 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:00.251872 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:00.251872 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:00.251872 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:00.251872 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:00.251872 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:00.251872 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:29:00.251872 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:29:00.251872 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:29:00.252765 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:29:00.252765 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:00.252765 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:00.252765 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:00.252765 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:00.252765 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:00.252765 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:00.252765 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:00.252765 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:00.252765 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:00.252765 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:00.252765 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:00.252765 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:00.252765 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:00.252765 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:00.252765 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:00.252765 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:00.254041 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:29:00.254041 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:29:00.254041 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:29:00.254041 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:29:00.254041 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:29:00.254041 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:29:00.254041 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:00.254041 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:00.254041 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 349/698] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:28:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:00.254785 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082df06d-f544-43d0-a376-c3fac48fe453 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:00.254938 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082df06d-f544-43d0-a376-c3fac48fe453 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:00.255166 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082df06d-f544-43d0-a376-c3fac48fe453 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:00.260898 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 350/699] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:00.261720 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082df06d-f544-43d0-a376-c3fac48fe453 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:00.301168 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ca2c00-c8f9-4dbb-991d-772d4ee05103 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:00.338602 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0ddfb-0391-4632-bcff-0caafc2ef99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:00.339686 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0ddfb-0391-4632-bcff-0caafc2ef99a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:00.340191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0ddfb-0391-4632-bcff-0caafc2ef99a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:00.340669 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0ddfb-0391-4632-bcff-0caafc2ef99a None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:00.341595 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e0ddfb-0391-4632-bcff-0caafc2ef99a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:00.342136 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e0ddfb-0391-4632-bcff-0caafc2ef99a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:00.343754 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e0ddfb-0391-4632-bcff-0caafc2ef99a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:00.357137 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 350/700] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:00.358635 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c55504c-3bcf-47a2-9bec-c666b07276e8 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:00.359213 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c55504c-3bcf-47a2-9bec-c666b07276e8 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:00.359700 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c55504c-3bcf-47a2-9bec-c666b07276e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:00.378950 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-2c55504c-3bcf-47a2-9bec-c666b07276e8 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:00.380500 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2c55504c-3bcf-47a2-9bec-c666b07276e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:00.386979 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082df06d-f544-43d0-a376-c3fac48fe453 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:00.395949 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:00.424636 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc323a3d-a248-459a-8763-8f4727bbd0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:00.425237 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc323a3d-a248-459a-8763-8f4727bbd0b0 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:00.425548 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc323a3d-a248-459a-8763-8f4727bbd0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:00.425821 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc323a3d-a248-459a-8763-8f4727bbd0b0 None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:00.426580 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc323a3d-a248-459a-8763-8f4727bbd0b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:00.426879 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc323a3d-a248-459a-8763-8f4727bbd0b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:00.428455 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc323a3d-a248-459a-8763-8f4727bbd0b0 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:00.442556 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 351/701] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 15:29:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:00.443294 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc323a3d-a248-459a-8763-8f4727bbd0b0 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:00.457672 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 351/702] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:29:00.458819 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c55504c-3bcf-47a2-9bec-c666b07276e8 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:00.473954 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:00.549377 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc323a3d-a248-459a-8763-8f4727bbd0b0 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:00.590218 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd7ed51-70f7-48cd-8a8f-d8e82240c611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:00.590810 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd7ed51-70f7-48cd-8a8f-d8e82240c611 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:00.591085 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd7ed51-70f7-48cd-8a8f-d8e82240c611 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:00.591351 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd7ed51-70f7-48cd-8a8f-d8e82240c611 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:00.603701 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd7ed51-70f7-48cd-8a8f-d8e82240c611 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:00.605236 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd7ed51-70f7-48cd-8a8f-d8e82240c611 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:00.616710 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-0dd7ed51-70f7-48cd-8a8f-d8e82240c611 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:00.616710 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:00.616710 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:00.616710 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:00.616710 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:00.616710 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:00.616710 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:00.616710 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:00.616710 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:00.616710 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:00.616710 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:00.616710 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:00.616710 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:29:00.616710 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:29:00.616710 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:29:00.618163 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:29:00.618163 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:00.618163 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:00.618163 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:00.618163 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:00.618163 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:00.618163 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:00.618163 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:00.618163 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:00.618163 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:00.618163 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:00.618163 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:00.618163 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:00.618163 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:00.618163 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:00.618163 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:00.618163 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:00.619989 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:29:00.619989 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:29:00.619989 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:29:00.619989 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:29:00.619989 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:29:00.619989 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:29:00.619989 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:00.619989 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:00.619989 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 352/703] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:29:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:00.623619 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd7ed51-70f7-48cd-8a8f-d8e82240c611 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:00.637840 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c55504c-3bcf-47a2-9bec-c666b07276e8 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:00.707491 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6b9b9c-6b8d-43c0-8177-d12a24458776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:00.708162 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b9b9c-6b8d-43c0-8177-d12a24458776 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:00.708395 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b9b9c-6b8d-43c0-8177-d12a24458776 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:00.708669 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b9b9c-6b8d-43c0-8177-d12a24458776 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:00.709393 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6b9b9c-6b8d-43c0-8177-d12a24458776 None admin] RBAC: Adding query filter params (role.id=62d96ac9513c452894fc67b442cfc59f, scope.project.id=3dc6c5f5fcd34922b0d09cd2aaed6cf1, user.id=8daf14b64bf04b2ba74d2fe1fd41ff71) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:00.709701 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6b9b9c-6b8d-43c0-8177-d12a24458776 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:00.711766 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6b9b9c-6b8d-43c0-8177-d12a24458776 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:00.732142 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 352/704] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 15:29:00 2022] GET /identity/v3/role_assignments?user.id=8daf14b64bf04b2ba74d2fe1fd41ff71&scope.project.id=3dc6c5f5fcd34922b0d09cd2aaed6cf1&role.id=62d96ac9513c452894fc67b442cfc59f => generated 259 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:00.736842 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:00.769042 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd7ed51-70f7-48cd-8a8f-d8e82240c611 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:00.824239 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f549cf-070f-46ba-b480-eb76a7eb8328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:00.824916 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f549cf-070f-46ba-b480-eb76a7eb8328 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:00.825249 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f549cf-070f-46ba-b480-eb76a7eb8328 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:00.825525 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f549cf-070f-46ba-b480-eb76a7eb8328 None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:00.826284 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f549cf-070f-46ba-b480-eb76a7eb8328 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:00.826592 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f549cf-070f-46ba-b480-eb76a7eb8328 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:00.828173 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f549cf-070f-46ba-b480-eb76a7eb8328 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:00.848080 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 353/705] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 15:29:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:00.854025 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6b9b9c-6b8d-43c0-8177-d12a24458776 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:00.854199 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f549cf-070f-46ba-b480-eb76a7eb8328 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:00.870238 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:00.978598 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6b9b9c-6b8d-43c0-8177-d12a24458776 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:00.996815 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f549cf-070f-46ba-b480-eb76a7eb8328 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:01.035316 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aee087-82c2-42bd-9d21-287294d5e751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:01.035952 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aee087-82c2-42bd-9d21-287294d5e751 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:01.036247 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aee087-82c2-42bd-9d21-287294d5e751 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:01.036527 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aee087-82c2-42bd-9d21-287294d5e751 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:01.037312 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aee087-82c2-42bd-9d21-287294d5e751 None admin] RBAC: Adding query filter params (role.id=342b64fe0cdb4925b23c6cd461a152ac, scope.project.id=3dc6c5f5fcd34922b0d09cd2aaed6cf1, user.id=328051fe7607464f8f5dc0dcc01a817a) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:01.037631 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aee087-82c2-42bd-9d21-287294d5e751 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:01.039888 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aee087-82c2-42bd-9d21-287294d5e751 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:01.043704 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bacc6e7-47a7-42d2-b014-83ea3e17c751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:01.044813 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bacc6e7-47a7-42d2-b014-83ea3e17c751 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:01.045234 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bacc6e7-47a7-42d2-b014-83ea3e17c751 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:01.045643 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bacc6e7-47a7-42d2-b014-83ea3e17c751 None admin] PATH_INFO: `/v3/services/network` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:01.046487 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bacc6e7-47a7-42d2-b014-83ea3e17c751 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:01.046602 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 353/706] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 15:29:00 2022] GET /identity/v3/role_assignments?user.id=328051fe7607464f8f5dc0dcc01a817a&scope.project.id=3dc6c5f5fcd34922b0d09cd2aaed6cf1&role.id=342b64fe0cdb4925b23c6cd461a152ac => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:01.047862 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af0521-18fb-471e-9578-c4ee07e8d292 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:01.048152 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af0521-18fb-471e-9578-c4ee07e8d292 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:01.048430 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af0521-18fb-471e-9578-c4ee07e8d292 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:01.048555 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bacc6e7-47a7-42d2-b014-83ea3e17c751 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:01.053885 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 354/707] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:01.055567 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be03e4b-3e36-4baa-8ce5-653eb829cb01 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:01.055852 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be03e4b-3e36-4baa-8ce5-653eb829cb01 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:01.056126 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be03e4b-3e36-4baa-8ce5-653eb829cb01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:01.080984 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-2bacc6e7-47a7-42d2-b014-83ea3e17c751 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 28 15:29:01.080984 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:01.080984 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:29:01.080984 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:29:01.080984 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:29:01.080984 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:29:01.080984 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:29:01.080984 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:29:01.080984 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 28 15:29:01.080984 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:01.080984 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:29:01.080984 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:01.080984 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:01.080984 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:01.080984 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:01.080984 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:01.080984 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:01.082112 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:01.082112 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:01.082112 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:01.082112 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:01.082112 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:01.082112 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:01.082112 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:29:01.082112 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:29:01.082112 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:29:01.082112 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:29:01.082112 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:01.082112 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:01.082112 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:01.082112 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:01.082112 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:01.082112 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:01.083005 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:01.083005 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:01.083005 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:01.083005 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:01.083005 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:01.083005 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:01.083005 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:01.083005 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:01.083005 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:01.083005 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:01.083005 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:29:01.083005 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:29:01.083005 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 28 15:29:01.083005 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:01.084184 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 354/708] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:29:00 2022] GET /identity/v3/services/network => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:01.088901 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bacc6e7-47a7-42d2-b014-83ea3e17c751 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:01.091006 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-8be03e4b-3e36-4baa-8ce5-653eb829cb01 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:01.092206 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8be03e4b-3e36-4baa-8ce5-653eb829cb01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:01.119360 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:01.213890 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 355/709] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:29:01.213890 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cad175-bd9c-4cdc-918e-025da03f4c9e None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:01.213890 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cad175-bd9c-4cdc-918e-025da03f4c9e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:01.213890 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cad175-bd9c-4cdc-918e-025da03f4c9e None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:01.214652 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 356/710] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:01.229641 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a5d08-4d40-4df7-9627-1c42b17bf8ba None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:01.229965 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a5d08-4d40-4df7-9627-1c42b17bf8ba None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:01.230152 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a5d08-4d40-4df7-9627-1c42b17bf8ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:01.241339 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bacc6e7-47a7-42d2-b014-83ea3e17c751 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:01.247611 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:01.250618 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-581a5d08-4d40-4df7-9627-1c42b17bf8ba None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:01.252011 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-581a5d08-4d40-4df7-9627-1c42b17bf8ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:01.288490 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e21c87a-0142-4b3e-ac0f-f5e8733de4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:01.290717 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e21c87a-0142-4b3e-ac0f-f5e8733de4d2 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:01.291195 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e21c87a-0142-4b3e-ac0f-f5e8733de4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:01.291656 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e21c87a-0142-4b3e-ac0f-f5e8733de4d2 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:01.292496 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e21c87a-0142-4b3e-ac0f-f5e8733de4d2 None admin] RBAC: Adding query filter params (name=network) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:01.293176 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e21c87a-0142-4b3e-ac0f-f5e8733de4d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:01.295178 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e21c87a-0142-4b3e-ac0f-f5e8733de4d2 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:01.308641 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 355/711] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Sep 28 15:29:01 2022] GET /identity/v3/services?name=network => generated 124 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:01.313720 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e21c87a-0142-4b3e-ac0f-f5e8733de4d2 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:01.350331 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 357/712] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 28 15:29:01.410353 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:01.435374 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e21c87a-0142-4b3e-ac0f-f5e8733de4d2 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:01.473986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef373df-cfbc-452e-a1a2-c9fa8f4fca46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:01.474592 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef373df-cfbc-452e-a1a2-c9fa8f4fca46 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:01.474871 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef373df-cfbc-452e-a1a2-c9fa8f4fca46 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:01.475198 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef373df-cfbc-452e-a1a2-c9fa8f4fca46 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:01.475833 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef373df-cfbc-452e-a1a2-c9fa8f4fca46 None admin] RBAC: Adding query filter params (type=network) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:01.476187 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef373df-cfbc-452e-a1a2-c9fa8f4fca46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:01.478208 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef373df-cfbc-452e-a1a2-c9fa8f4fca46 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:01.484193 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 356/713] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Sep 28 15:29:01 2022] GET /identity/v3/services?type=network => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:01.525448 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581a5d08-4d40-4df7-9627-1c42b17bf8ba None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:01.638910 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581a5d08-4d40-4df7-9627-1c42b17bf8ba None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:01.646371 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef373df-cfbc-452e-a1a2-c9fa8f4fca46 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:01.696580 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9583262f-cfca-4283-827a-7ce2638ddb4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:01.697645 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9583262f-cfca-4283-827a-7ce2638ddb4f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:01.698150 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9583262f-cfca-4283-827a-7ce2638ddb4f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:01.698615 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9583262f-cfca-4283-827a-7ce2638ddb4f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:01.699385 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9583262f-cfca-4283-827a-7ce2638ddb4f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:01.700940 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9583262f-cfca-4283-827a-7ce2638ddb4f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:01.704879 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:01.713959 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-9583262f-cfca-4283-827a-7ce2638ddb4f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 28 15:29:01.713959 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:01.713959 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:29:01.713959 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:29:01.713959 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:29:01.713959 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:29:01.713959 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:29:01.713959 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:29:01.713959 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 28 15:29:01.713959 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:01.713959 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:29:01.713959 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:01.713959 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:01.713959 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:01.713959 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:01.713959 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:01.713959 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:01.715016 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:01.715016 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:01.715016 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:01.715016 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:01.715016 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:01.715016 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:01.715016 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:29:01.715016 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:29:01.715016 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:29:01.715016 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:29:01.715016 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:01.715016 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:01.715016 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:01.715016 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:01.715016 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:01.715016 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:01.715857 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:01.715857 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:01.715857 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:01.715857 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:01.715857 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:01.715857 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:01.715857 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:01.715857 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:01.715857 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:01.715857 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:01.715857 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:29:01.715857 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:29:01.715857 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 28 15:29:01.715857 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:01.718613 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 358/714] 10.222.0.52 () {56 vars in 1129 bytes} [Wed Sep 28 15:29:01 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:01.731502 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9583262f-cfca-4283-827a-7ce2638ddb4f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:01.824938 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef373df-cfbc-452e-a1a2-c9fa8f4fca46 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:01.854405 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9583262f-cfca-4283-827a-7ce2638ddb4f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:01.899473 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b7618b-ae92-450f-b263-0e1826034a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:01.900105 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7618b-ae92-450f-b263-0e1826034a61 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:01.900383 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7618b-ae92-450f-b263-0e1826034a61 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:01.900650 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7618b-ae92-450f-b263-0e1826034a61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:01.916869 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65417726-c4be-44eb-bc9a-0940dc924675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:01.917545 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65417726-c4be-44eb-bc9a-0940dc924675 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:01.917833 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65417726-c4be-44eb-bc9a-0940dc924675 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:01.918129 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65417726-c4be-44eb-bc9a-0940dc924675 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:01.918755 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65417726-c4be-44eb-bc9a-0940dc924675 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:01.919058 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65417726-c4be-44eb-bc9a-0940dc924675 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:01.919585 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:01.922659 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65417726-c4be-44eb-bc9a-0940dc924675 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:01.924560 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b7618b-ae92-450f-b263-0e1826034a61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:01.926306 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b7618b-ae92-450f-b263-0e1826034a61 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:01.933421 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 359/715] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Sep 28 15:29:01 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:01.938618 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65417726-c4be-44eb-bc9a-0940dc924675 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:01.959758 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-e8b7618b-ae92-450f-b263-0e1826034a61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:01.959758 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:01.959758 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:01.959758 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:01.959758 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:01.959758 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:01.959758 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:01.959758 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:01.959758 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:01.959758 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:01.959758 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:01.959758 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:01.959758 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:01.959758 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:01.959758 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:01.959758 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:01.960861 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:01.960861 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:01.960861 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:01.960861 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:01.960861 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:01.960861 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:01.960861 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:01.960861 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:01.960861 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:01.960861 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:01.960861 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:01.960861 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:01.960861 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:01.960861 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:01.960861 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:01.960861 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:01.960861 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:01.963217 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:01.963217 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:01.963217 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:01.963217 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 357/716] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:01.970475 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b7618b-ae92-450f-b263-0e1826034a61 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:02.056313 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:02.133673 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65417726-c4be-44eb-bc9a-0940dc924675 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:02.170191 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b7618b-ae92-450f-b263-0e1826034a61 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:02.176840 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa73de-1986-4f09-9896-ff502a043fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:02.177452 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa73de-1986-4f09-9896-ff502a043fa6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:02.177731 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa73de-1986-4f09-9896-ff502a043fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:02.178077 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa73de-1986-4f09-9896-ff502a043fa6 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:02.178772 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fa73de-1986-4f09-9896-ff502a043fa6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:02.179078 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fa73de-1986-4f09-9896-ff502a043fa6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:02.180652 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fa73de-1986-4f09-9896-ff502a043fa6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:02.186691 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 360/717] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Sep 28 15:29:01 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:02.241630 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2fd62a-e223-4e4f-8172-4ad7aae7999e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:02.242818 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2fd62a-e223-4e4f-8172-4ad7aae7999e None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:02.243098 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2fd62a-e223-4e4f-8172-4ad7aae7999e None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:02.243370 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2fd62a-e223-4e4f-8172-4ad7aae7999e None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:02.244023 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2fd62a-e223-4e4f-8172-4ad7aae7999e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:02.244323 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2fd62a-e223-4e4f-8172-4ad7aae7999e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:02.245702 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2fd62a-e223-4e4f-8172-4ad7aae7999e None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:02.259758 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 358/718] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:02.267295 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa73de-1986-4f09-9896-ff502a043fa6 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:02.421368 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa73de-1986-4f09-9896-ff502a043fa6 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:02.453045 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:02.461292 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d5d547-294c-4a63-ba13-47f26dc0f3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:02.462260 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d5d547-294c-4a63-ba13-47f26dc0f3a1 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:02.462745 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d5d547-294c-4a63-ba13-47f26dc0f3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:02.463351 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d5d547-294c-4a63-ba13-47f26dc0f3a1 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:02.464002 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d5d547-294c-4a63-ba13-47f26dc0f3a1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:02.465434 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d5d547-294c-4a63-ba13-47f26dc0f3a1 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:02.488658 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 361/719] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 15:29:02 2022] POST /identity/v3/users => generated 254 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:29:03.214786 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684f47f-8115-4210-bb13-d8b0855c82e3 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:03.215814 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684f47f-8115-4210-bb13-d8b0855c82e3 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:03.216229 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684f47f-8115-4210-bb13-d8b0855c82e3 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:03.217944 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 359/720] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:03.223435 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a70e8-dc11-45ad-9746-abd3bf713fb6 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:03.223724 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a70e8-dc11-45ad-9746-abd3bf713fb6 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:03.224001 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a70e8-dc11-45ad-9746-abd3bf713fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:03.250910 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d23a70e8-dc11-45ad-9746-abd3bf713fb6 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:03.252178 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d23a70e8-dc11-45ad-9746-abd3bf713fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:03.350577 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 362/721] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:29:03.745441 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8684f47f-8115-4210-bb13-d8b0855c82e3 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:03.815017 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:03.888585 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8684f47f-8115-4210-bb13-d8b0855c82e3 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:03.936875 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40bdf1a-e334-44a0-9dd8-355ecf1f9733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:03.937505 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40bdf1a-e334-44a0-9dd8-355ecf1f9733 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:03.937782 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40bdf1a-e334-44a0-9dd8-355ecf1f9733 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:03.938081 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40bdf1a-e334-44a0-9dd8-355ecf1f9733 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:03.956436 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40bdf1a-e334-44a0-9dd8-355ecf1f9733 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:03.957371 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ffe999-2465-4fde-a863-4bbce6575190 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:03.957882 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ffe999-2465-4fde-a863-4bbce6575190 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:03.958351 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ffe999-2465-4fde-a863-4bbce6575190 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:03.961773 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 363/722] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:03.961906 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-c40bdf1a-e334-44a0-9dd8-355ecf1f9733 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:29:03.961906 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:03.961906 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:03.961906 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:03.961906 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:03.961906 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:03.961906 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:03.961906 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:03.961906 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:03.961906 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:03.961906 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:03.961906 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:03.961906 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:03.961906 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:03.961906 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:03.961906 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:03.962786 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:03.962786 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:03.962786 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:03.962786 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:03.962786 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:03.962786 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:03.962786 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:03.962786 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:03.962786 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:03.962786 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:03.962786 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:03.962786 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:03.962786 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:03.962786 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:03.962786 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:03.962786 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:03.964223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:03.964223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:03.964223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:03.964223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:03.964223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:03.964223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:03.964223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:03.964223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:03.964223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:03.964223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:03.964223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:29:03.964223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:03.964223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 360/723] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 15:29:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:03.968464 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40bdf1a-e334-44a0-9dd8-355ecf1f9733 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:03.969771 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78297b1-3831-4f6a-80bf-17f858dd1d53 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:03.970265 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78297b1-3831-4f6a-80bf-17f858dd1d53 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:03.970372 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78297b1-3831-4f6a-80bf-17f858dd1d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:03.989002 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d78297b1-3831-4f6a-80bf-17f858dd1d53 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:03.990196 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d78297b1-3831-4f6a-80bf-17f858dd1d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:04.099839 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 364/724] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:29:04.129362 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40bdf1a-e334-44a0-9dd8-355ecf1f9733 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:04.144835 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:04.166583 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebdd24e-ce08-4a68-81cc-5dca21aad41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:04.167187 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebdd24e-ce08-4a68-81cc-5dca21aad41a None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:04.167468 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebdd24e-ce08-4a68-81cc-5dca21aad41a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:04.167743 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebdd24e-ce08-4a68-81cc-5dca21aad41a None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:04.168400 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebdd24e-ce08-4a68-81cc-5dca21aad41a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:04.168704 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebdd24e-ce08-4a68-81cc-5dca21aad41a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:04.170169 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebdd24e-ce08-4a68-81cc-5dca21aad41a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:04.178654 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 361/725] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:29:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:04.183896 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78297b1-3831-4f6a-80bf-17f858dd1d53 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:04.243460 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6014fe1-e4bb-4f57-b56d-1c1a408cdaf1 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:04.243980 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6014fe1-e4bb-4f57-b56d-1c1a408cdaf1 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:04.243980 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6014fe1-e4bb-4f57-b56d-1c1a408cdaf1 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:04.245311 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 362/726] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:29:04.251226 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:04.255051 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e80a7a-5564-4992-b1e3-535d19c12727 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:04.255819 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e80a7a-5564-4992-b1e3-535d19c12727 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:04.256300 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e80a7a-5564-4992-b1e3-535d19c12727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:04.281279 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-91e80a7a-5564-4992-b1e3-535d19c12727 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:04.282862 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-91e80a7a-5564-4992-b1e3-535d19c12727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:04.325667 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78297b1-3831-4f6a-80bf-17f858dd1d53 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:04.356981 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 363/727] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:29:04.375586 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04af455c-33b9-4f67-af60-4e72dc63184b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:04.376844 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af455c-33b9-4f67-af60-4e72dc63184b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:04.377359 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af455c-33b9-4f67-af60-4e72dc63184b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:04.377834 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af455c-33b9-4f67-af60-4e72dc63184b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:04.388380 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04af455c-33b9-4f67-af60-4e72dc63184b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:04.390110 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04af455c-33b9-4f67-af60-4e72dc63184b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:04.405064 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-04af455c-33b9-4f67-af60-4e72dc63184b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:04.405064 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:04.405064 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:04.405064 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:04.405064 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:04.405064 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:04.405064 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:04.405064 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:04.405064 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:04.405064 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:04.405064 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:04.405064 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:04.405064 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:04.405064 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:04.405064 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:04.405064 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:04.406080 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:04.406080 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:04.406080 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:04.406080 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:04.406080 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:04.406080 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:04.406080 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:04.406080 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:04.406080 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:04.406080 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:04.406080 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:04.406080 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:04.406080 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:04.406080 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:04.406080 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:04.406080 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:04.406080 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:04.406988 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:04.406988 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:04.406988 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:04.409065 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 365/728] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:04.414835 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e80a7a-5564-4992-b1e3-535d19c12727 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:04.427066 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04af455c-33b9-4f67-af60-4e72dc63184b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:04.575644 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e80a7a-5564-4992-b1e3-535d19c12727 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:04.578019 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:04.611980 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:04.622371 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04af455c-33b9-4f67-af60-4e72dc63184b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:04.639947 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd84f67f-7913-468a-b008-ea301bf8cf6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:04.640550 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd84f67f-7913-468a-b008-ea301bf8cf6a None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:04.640828 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd84f67f-7913-468a-b008-ea301bf8cf6a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:04.641090 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd84f67f-7913-468a-b008-ea301bf8cf6a None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:04.641717 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd84f67f-7913-468a-b008-ea301bf8cf6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:04.642034 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd84f67f-7913-468a-b008-ea301bf8cf6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:04.643380 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd84f67f-7913-468a-b008-ea301bf8cf6a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:04.666957 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 364/729] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:04.670885 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd84f67f-7913-468a-b008-ea301bf8cf6a None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:04.686376 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae88551-1170-4279-a9fb-c606dfc95cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:04.687018 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae88551-1170-4279-a9fb-c606dfc95cf2 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:04.687305 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae88551-1170-4279-a9fb-c606dfc95cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:04.687579 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae88551-1170-4279-a9fb-c606dfc95cf2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:04.714049 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae88551-1170-4279-a9fb-c606dfc95cf2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:04.720939 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-cae88551-1170-4279-a9fb-c606dfc95cf2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:29:04.720939 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:04.720939 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:04.720939 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:04.720939 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:04.720939 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:04.720939 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:04.720939 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:04.720939 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:04.720939 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:04.720939 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:04.720939 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:04.720939 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:04.720939 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:04.720939 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:04.720939 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:04.722245 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:04.722245 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:04.722245 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:04.722245 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:04.722245 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:04.722245 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:04.722245 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:04.722245 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:04.722245 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:04.722245 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:04.722245 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:04.722245 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:04.722245 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:04.722245 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:04.722245 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:04.722245 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:04.723314 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:04.723314 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:04.723314 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:04.723314 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:04.723314 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:04.723314 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:04.723314 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:04.723314 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:04.723314 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:04.723314 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:04.723314 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:29:04.723314 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:04.723314 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 366/730] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 15:29:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:04.723314 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae88551-1170-4279-a9fb-c606dfc95cf2 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:04.797125 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd84f67f-7913-468a-b008-ea301bf8cf6a None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:04.838094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae88551-1170-4279-a9fb-c606dfc95cf2 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:04.839020 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14171da5-2835-4b6f-821b-44c78da1afc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:04.839647 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14171da5-2835-4b6f-821b-44c78da1afc8 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:04.839920 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14171da5-2835-4b6f-821b-44c78da1afc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:04.840181 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14171da5-2835-4b6f-821b-44c78da1afc8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:04.853684 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14171da5-2835-4b6f-821b-44c78da1afc8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:04.855775 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14171da5-2835-4b6f-821b-44c78da1afc8 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:04.870554 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-14171da5-2835-4b6f-821b-44c78da1afc8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 28 15:29:04.870554 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:04.870554 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:04.870554 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:04.870554 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:04.870554 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:04.870554 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:04.870554 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:04.870554 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:04.870554 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:04.870554 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:04.870554 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:04.870554 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:29:04.870554 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:29:04.870554 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:29:04.870554 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:29:04.871700 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:04.871700 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:04.871700 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:29:04.871700 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:04.871700 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:29:04.871700 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:04.871700 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:04.871700 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:04.871700 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:04.871700 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:04.871700 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:04.871700 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:04.871700 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:04.871700 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:04.871700 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:04.871700 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:04.871700 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:04.871700 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:04.872817 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:04.872817 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:04.872817 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:29:04.872817 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:29:04.872817 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:29:04.872817 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:29:04.872817 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:29:04.872817 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:29:04.872817 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 28 15:29:04.872817 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:04.872817 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 365/731] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Sep 28 15:29:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:04.872817 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14171da5-2835-4b6f-821b-44c78da1afc8 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:04.884147 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a75650e-04e3-477a-bda1-46be3e2e1547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:04.884771 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75650e-04e3-477a-bda1-46be3e2e1547 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:04.885511 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75650e-04e3-477a-bda1-46be3e2e1547 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:04.885790 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75650e-04e3-477a-bda1-46be3e2e1547 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:04.887055 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a75650e-04e3-477a-bda1-46be3e2e1547 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:04.888684 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a75650e-04e3-477a-bda1-46be3e2e1547 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:04.902671 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:04.914437 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 367/732] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Sep 28 15:29:04 2022] POST /identity/v3/services => generated 240 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:29:04.915570 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a75650e-04e3-477a-bda1-46be3e2e1547 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:04.962226 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:05.038577 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14171da5-2835-4b6f-821b-44c78da1afc8 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:05.103176 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459407b2-a2fc-4d28-bdc6-debfcce9eab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:05.103791 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459407b2-a2fc-4d28-bdc6-debfcce9eab2 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:05.104066 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459407b2-a2fc-4d28-bdc6-debfcce9eab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:05.104331 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459407b2-a2fc-4d28-bdc6-debfcce9eab2 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:05.105073 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459407b2-a2fc-4d28-bdc6-debfcce9eab2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:05.105383 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459407b2-a2fc-4d28-bdc6-debfcce9eab2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:05.106851 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459407b2-a2fc-4d28-bdc6-debfcce9eab2 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:05.115561 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a75650e-04e3-477a-bda1-46be3e2e1547 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:05.120897 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 366/733] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:29:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:05.120897 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c3596c-48ea-40df-b52f-1960e05e7f7a None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:05.120897 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c3596c-48ea-40df-b52f-1960e05e7f7a None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:05.120897 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c3596c-48ea-40df-b52f-1960e05e7f7a None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:05.120897 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 367/734] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:05.121821 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c3596c-48ea-40df-b52f-1960e05e7f7a None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:05.164084 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c880dc-d496-4891-a531-8dee86736c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:05.164715 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c880dc-d496-4891-a531-8dee86736c26 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:05.164976 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c880dc-d496-4891-a531-8dee86736c26 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:05.165249 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c880dc-d496-4891-a531-8dee86736c26 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:05.166085 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c880dc-d496-4891-a531-8dee86736c26 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:05.166310 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c880dc-d496-4891-a531-8dee86736c26 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:05.167740 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c880dc-d496-4891-a531-8dee86736c26 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:05.191221 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 368/735] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Sep 28 15:29:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:05.193008 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151079d3-77e6-46eb-8442-13b34784d49b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:05.193309 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151079d3-77e6-46eb-8442-13b34784d49b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:05.193582 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151079d3-77e6-46eb-8442-13b34784d49b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:05.219138 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-151079d3-77e6-46eb-8442-13b34784d49b None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:05.221504 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-151079d3-77e6-46eb-8442-13b34784d49b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:05.238989 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:05.264837 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c3596c-48ea-40df-b52f-1960e05e7f7a None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:05.318703 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558ac47b-734d-4f2a-9d2a-961dcc241385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:05.319648 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ac47b-734d-4f2a-9d2a-961dcc241385 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:05.319868 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 369/736] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:05 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:29:05.326134 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151079d3-77e6-46eb-8442-13b34784d49b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:05.326351 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ac47b-734d-4f2a-9d2a-961dcc241385 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:05.326589 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ac47b-734d-4f2a-9d2a-961dcc241385 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:05.329054 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:05.350127 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558ac47b-734d-4f2a-9d2a-961dcc241385 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:05.352541 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558ac47b-734d-4f2a-9d2a-961dcc241385 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:05.383885 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-558ac47b-734d-4f2a-9d2a-961dcc241385 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:05.383885 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:05.383885 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:05.383885 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:05.383885 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:05.383885 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:05.383885 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:05.383885 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:05.383885 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:05.383885 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:05.383885 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:05.383885 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:05.383885 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:05.383885 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:05.383885 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:05.383885 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:05.384836 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:05.384836 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:05.384836 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:05.384836 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:05.384836 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:05.384836 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:05.384836 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:05.384836 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:05.384836 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:05.384836 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:05.384836 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:05.384836 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:05.384836 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:05.384836 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:05.384836 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:05.384836 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:05.384836 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:05.385749 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:05.385749 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:05.385749 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:05.385749 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 368/737] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:05.396670 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558ac47b-734d-4f2a-9d2a-961dcc241385 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:05.503618 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151079d3-77e6-46eb-8442-13b34784d49b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:05.539345 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c638ac84-57c8-4a57-82ee-6a97b65bbee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:05.539977 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c638ac84-57c8-4a57-82ee-6a97b65bbee3 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:05.540261 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c638ac84-57c8-4a57-82ee-6a97b65bbee3 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:05.540523 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558ac47b-734d-4f2a-9d2a-961dcc241385 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:05.540624 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c638ac84-57c8-4a57-82ee-6a97b65bbee3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:05.551045 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c638ac84-57c8-4a57-82ee-6a97b65bbee3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:05.552547 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c638ac84-57c8-4a57-82ee-6a97b65bbee3 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:05.572453 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-c638ac84-57c8-4a57-82ee-6a97b65bbee3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:05.572453 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:05.572453 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:05.572453 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:05.572453 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:05.572453 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:05.572453 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:05.572453 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:05.572453 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:05.572453 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:05.572453 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:05.572453 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:05.572453 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:05.572453 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:05.572453 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:05.572453 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:05.573624 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:05.573624 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:05.573624 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:05.573624 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:05.573624 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:05.573624 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:05.573624 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:05.573624 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:05.573624 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:05.573624 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:05.573624 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:05.573624 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:05.573624 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:05.573624 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:05.573624 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:05.573624 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:05.573624 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:05.574772 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:05.574772 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:05.574772 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:05.574772 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 370/738] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:05.575215 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272bae01-01e5-4e32-90eb-fb4668251bf0 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:05.576247 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272bae01-01e5-4e32-90eb-fb4668251bf0 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:05.576247 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272bae01-01e5-4e32-90eb-fb4668251bf0 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:05.578660 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 371/739] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:05.581136 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272bae01-01e5-4e32-90eb-fb4668251bf0 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:05.587191 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc0b769-c512-4e77-92f2-ac5eba374787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:05.587967 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc0b769-c512-4e77-92f2-ac5eba374787 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:05.588250 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc0b769-c512-4e77-92f2-ac5eba374787 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:05.588521 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc0b769-c512-4e77-92f2-ac5eba374787 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:05.589170 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc0b769-c512-4e77-92f2-ac5eba374787 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:05.589479 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc0b769-c512-4e77-92f2-ac5eba374787 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:05.591013 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc0b769-c512-4e77-92f2-ac5eba374787 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:05.602173 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:05.605821 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 369/740] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:05.607041 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67132e17-3f64-42e9-ad3a-7e4eb0e60003 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:05.607324 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67132e17-3f64-42e9-ad3a-7e4eb0e60003 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:05.607594 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67132e17-3f64-42e9-ad3a-7e4eb0e60003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:05.647753 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:05.700146 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272bae01-01e5-4e32-90eb-fb4668251bf0 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:05.737059 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f5759-d556-45b3-8337-951a254b5d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:05.738114 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f5759-d556-45b3-8337-951a254b5d5d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:05.738589 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f5759-d556-45b3-8337-951a254b5d5d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:05.739052 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731f5759-d556-45b3-8337-951a254b5d5d None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:05.739908 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f5759-d556-45b3-8337-951a254b5d5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:05.740400 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f5759-d556-45b3-8337-951a254b5d5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:05.742118 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731f5759-d556-45b3-8337-951a254b5d5d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:05.750397 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-67132e17-3f64-42e9-ad3a-7e4eb0e60003 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:05.751619 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-67132e17-3f64-42e9-ad3a-7e4eb0e60003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:05.760765 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 372/741] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:05.761918 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f5759-d556-45b3-8337-951a254b5d5d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:05.827061 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 370/742] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:05 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 28 15:29:05.828641 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67132e17-3f64-42e9-ad3a-7e4eb0e60003 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:05.888860 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731f5759-d556-45b3-8337-951a254b5d5d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:05.908191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:05.936423 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecbaab1-d21c-4bb8-959e-40127359d41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:05.937160 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecbaab1-d21c-4bb8-959e-40127359d41f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:05.937550 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecbaab1-d21c-4bb8-959e-40127359d41f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:05.937948 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecbaab1-d21c-4bb8-959e-40127359d41f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:05.945202 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67132e17-3f64-42e9-ad3a-7e4eb0e60003 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:05.948357 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecbaab1-d21c-4bb8-959e-40127359d41f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:05.950062 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecbaab1-d21c-4bb8-959e-40127359d41f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:05.964916 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-8ecbaab1-d21c-4bb8-959e-40127359d41f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 28 15:29:05.964916 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:05.964916 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:05.964916 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:05.964916 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:05.964916 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:05.964916 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:05.964916 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:05.964916 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:05.964916 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:05.964916 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:05.964916 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:05.964916 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:29:05.964916 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:29:05.964916 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:29:05.964916 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:29:05.966186 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:05.966186 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:05.966186 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:29:05.966186 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:05.966186 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:29:05.966186 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:05.966186 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:05.966186 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:05.966186 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:05.966186 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:05.966186 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:05.966186 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:05.966186 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:05.966186 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:05.966186 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:05.966186 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:05.966186 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:05.966186 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:05.967178 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:05.967178 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:05.967178 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:29:05.967178 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:29:05.967178 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:29:05.967178 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:29:05.967178 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:29:05.967178 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:29:05.967178 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 28 15:29:05.967178 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:05.967178 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 373/743] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 15:29:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:05.967178 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecbaab1-d21c-4bb8-959e-40127359d41f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:05.974783 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:05.988382 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b505e3c3-65ad-4e28-adc8-764475e80864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:05.989290 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b505e3c3-65ad-4e28-adc8-764475e80864 None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:05.989818 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b505e3c3-65ad-4e28-adc8-764475e80864 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:05.990342 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b505e3c3-65ad-4e28-adc8-764475e80864 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:05.991237 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b505e3c3-65ad-4e28-adc8-764475e80864 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:05.993390 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b505e3c3-65ad-4e28-adc8-764475e80864 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:06.014775 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 371/744] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Sep 28 15:29:05 2022] POST /identity/v3/services => generated 251 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:29:06.016026 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b505e3c3-65ad-4e28-adc8-764475e80864 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:06.155706 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecbaab1-d21c-4bb8-959e-40127359d41f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:06.169696 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b505e3c3-65ad-4e28-adc8-764475e80864 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:06.197671 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c76b791-57bd-4b0d-869c-7b3aea6951c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:06.198816 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c76b791-57bd-4b0d-869c-7b3aea6951c6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:06.199369 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c76b791-57bd-4b0d-869c-7b3aea6951c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:06.199989 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c76b791-57bd-4b0d-869c-7b3aea6951c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:06.206105 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926e31a1-6fb5-4b5e-811c-5b76ed21428a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:06.206713 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e31a1-6fb5-4b5e-811c-5b76ed21428a None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:06.206985 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e31a1-6fb5-4b5e-811c-5b76ed21428a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:06.207251 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e31a1-6fb5-4b5e-811c-5b76ed21428a None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:06.207982 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926e31a1-6fb5-4b5e-811c-5b76ed21428a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:06.208281 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926e31a1-6fb5-4b5e-811c-5b76ed21428a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:06.209332 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c76b791-57bd-4b0d-869c-7b3aea6951c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:06.209698 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926e31a1-6fb5-4b5e-811c-5b76ed21428a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:06.211980 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c76b791-57bd-4b0d-869c-7b3aea6951c6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:06.227100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-9c76b791-57bd-4b0d-869c-7b3aea6951c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:06.227100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:06.227100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:06.227100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:06.227100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:06.227100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:06.227100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:06.227100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:06.227100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:06.227100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:06.227100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:06.227100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:06.227100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:29:06.227100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:29:06.227100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:29:06.228045 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:29:06.228045 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:06.228045 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:06.228045 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:06.228045 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:06.228045 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:06.228045 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:06.228045 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:06.228045 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:06.228045 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:06.228045 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:06.228045 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:06.228045 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:06.228045 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:06.228045 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:06.228045 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:06.228045 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:06.228966 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:29:06.228966 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:29:06.228966 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:29:06.228966 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:29:06.228966 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:29:06.228966 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:29:06.228966 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:06.228966 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:06.229455 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 372/745] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Sep 28 15:29:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:06.229998 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926e31a1-6fb5-4b5e-811c-5b76ed21428a None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:06.231780 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 374/746] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:29:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:06.234866 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c76b791-57bd-4b0d-869c-7b3aea6951c6 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:06.289729 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:06.326285 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:06.343246 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926e31a1-6fb5-4b5e-811c-5b76ed21428a None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:06.383057 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345bc0b0-af54-4107-bc7c-21cc5901d545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:06.383491 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c76b791-57bd-4b0d-869c-7b3aea6951c6 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:06.383808 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345bc0b0-af54-4107-bc7c-21cc5901d545 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:06.384159 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345bc0b0-af54-4107-bc7c-21cc5901d545 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:06.384430 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345bc0b0-af54-4107-bc7c-21cc5901d545 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:06.402070 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345bc0b0-af54-4107-bc7c-21cc5901d545 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:06.407927 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-345bc0b0-af54-4107-bc7c-21cc5901d545 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:29:06.407927 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:06.407927 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:06.407927 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:06.407927 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:06.407927 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:06.407927 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:06.407927 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:06.407927 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:06.407927 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:06.407927 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:06.407927 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:06.407927 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:06.407927 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:06.407927 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:06.407927 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:06.409632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:06.409632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:06.409632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:06.409632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:06.409632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:06.409632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:06.409632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:06.409632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:06.409632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:06.409632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:06.409632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:06.409632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:06.409632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:06.409632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:06.409632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:06.409632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:06.410779 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:06.410779 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:06.410779 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:06.410779 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:06.410779 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:06.410779 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:06.410779 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:06.410779 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:06.410779 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:06.410779 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:06.410779 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:29:06.410779 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:06.410779 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 373/747] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 15:29:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:06.410779 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345bc0b0-af54-4107-bc7c-21cc5901d545 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:06.425618 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0854ad00-881b-4f5f-bf87-fff3e6ef0543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:06.426384 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0854ad00-881b-4f5f-bf87-fff3e6ef0543 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:06.426755 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0854ad00-881b-4f5f-bf87-fff3e6ef0543 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:06.427120 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0854ad00-881b-4f5f-bf87-fff3e6ef0543 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:06.515659 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0854ad00-881b-4f5f-bf87-fff3e6ef0543 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:06.517442 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0854ad00-881b-4f5f-bf87-fff3e6ef0543 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:06.524919 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:06.531702 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345bc0b0-af54-4107-bc7c-21cc5901d545 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:06.556605 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:06.580287 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f87750e-81e7-4db2-801f-ee3708934830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:06.581545 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87750e-81e7-4db2-801f-ee3708934830 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:06.582358 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87750e-81e7-4db2-801f-ee3708934830 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:06.582795 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87750e-81e7-4db2-801f-ee3708934830 None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:06.584324 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f87750e-81e7-4db2-801f-ee3708934830 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:06.585040 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f87750e-81e7-4db2-801f-ee3708934830 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:06.587323 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f87750e-81e7-4db2-801f-ee3708934830 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:06.603443 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 374/748] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 15:29:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:06.604870 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f87750e-81e7-4db2-801f-ee3708934830 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:06.642234 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 375/749] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:29:06 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 28 15:29:06.642234 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0854ad00-881b-4f5f-bf87-fff3e6ef0543 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:06.745419 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0854ad00-881b-4f5f-bf87-fff3e6ef0543 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:06.746306 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f87750e-81e7-4db2-801f-ee3708934830 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:06.785622 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33139db8-dd17-479d-91c4-ea15bf998577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:06.786379 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33139db8-dd17-479d-91c4-ea15bf998577 None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:06.786658 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33139db8-dd17-479d-91c4-ea15bf998577 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:06.786924 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33139db8-dd17-479d-91c4-ea15bf998577 None admin] PATH_INFO: `/v3/projects/3dc6c5f5fcd34922b0d09cd2aaed6cf1/users/8daf14b64bf04b2ba74d2fe1fd41ff71/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:06.788842 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33139db8-dd17-479d-91c4-ea15bf998577 None admin] RBAC: Authorizing `identity:create_grant(project_id=3dc6c5f5fcd34922b0d09cd2aaed6cf1, user_id=8daf14b64bf04b2ba74d2fe1fd41ff71, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:06.790471 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33139db8-dd17-479d-91c4-ea15bf998577 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:06.793674 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:06.794568 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31196cad-e0b1-4651-b181-0147f54d0e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:06.795587 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31196cad-e0b1-4651-b181-0147f54d0e2a None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:06.796053 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31196cad-e0b1-4651-b181-0147f54d0e2a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:06.796786 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31196cad-e0b1-4651-b181-0147f54d0e2a None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:06.797733 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31196cad-e0b1-4651-b181-0147f54d0e2a None admin] RBAC: Adding query filter params (name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:06.798437 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31196cad-e0b1-4651-b181-0147f54d0e2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:06.800613 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31196cad-e0b1-4651-b181-0147f54d0e2a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:06.812390 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 375/750] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 15:29:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:06.813905 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31196cad-e0b1-4651-b181-0147f54d0e2a None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:06.815822 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 376/751] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 15:29:06 2022] PUT /identity/v3/projects/3dc6c5f5fcd34922b0d09cd2aaed6cf1/users/8daf14b64bf04b2ba74d2fe1fd41ff71/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:29:06.818039 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33139db8-dd17-479d-91c4-ea15bf998577 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:06.840923 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:06.932925 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33139db8-dd17-479d-91c4-ea15bf998577 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:06.933131 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31196cad-e0b1-4651-b181-0147f54d0e2a None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:06.970945 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86349dca-cfa8-433d-a799-e36284d4b9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:06.971972 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86349dca-cfa8-433d-a799-e36284d4b9d0 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:06.972486 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86349dca-cfa8-433d-a799-e36284d4b9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:06.972972 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86349dca-cfa8-433d-a799-e36284d4b9d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:06.985756 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86349dca-cfa8-433d-a799-e36284d4b9d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:06.987812 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86349dca-cfa8-433d-a799-e36284d4b9d0 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:06.999992 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1970c63c-e13c-43f9-b6a6-0211d8d5d5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:07.000641 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970c63c-e13c-43f9-b6a6-0211d8d5d5bb None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:07.000925 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970c63c-e13c-43f9-b6a6-0211d8d5d5bb None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:07.001229 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970c63c-e13c-43f9-b6a6-0211d8d5d5bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:07.006043 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-86349dca-cfa8-433d-a799-e36284d4b9d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:07.006043 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:07.006043 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:07.006043 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:07.006043 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:07.006043 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:07.006043 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:07.006043 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:07.006043 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:07.006043 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:07.006043 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:07.006043 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:07.006043 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:07.006043 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:07.006043 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:07.006043 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:07.006957 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:07.006957 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:07.006957 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:07.006957 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:07.006957 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:07.006957 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:07.006957 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:07.006957 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:07.006957 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:07.006957 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:07.006957 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:07.006957 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:07.006957 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:07.006957 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:07.006957 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:07.006957 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:07.006957 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:07.008435 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:07.008435 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:07.008435 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:07.010579 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 376/752] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:07.016284 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86349dca-cfa8-433d-a799-e36284d4b9d0 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:07.021168 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1970c63c-e13c-43f9-b6a6-0211d8d5d5bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:07.022149 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1970c63c-e13c-43f9-b6a6-0211d8d5d5bb None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:07.065370 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-1970c63c-e13c-43f9-b6a6-0211d8d5d5bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:07.065370 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:07.065370 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:07.065370 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:07.065370 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:07.065370 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:07.065370 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:07.065370 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:07.065370 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:07.065370 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:07.065370 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:07.065370 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:07.065370 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:07.065370 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:07.065370 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:07.065370 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:07.066408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:07.066408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:07.066408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:07.066408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:07.066408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:07.066408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:07.066408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:07.066408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:07.066408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:07.066408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:07.066408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:07.066408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:07.066408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:07.066408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:07.066408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:07.066408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:07.066408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:07.067295 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:07.067295 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:07.067295 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:07.067295 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 377/753] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:07.075045 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1970c63c-e13c-43f9-b6a6-0211d8d5d5bb None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:07.157446 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:07.164763 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:07.205779 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86349dca-cfa8-433d-a799-e36284d4b9d0 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:07.244117 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ee787f-8946-4bb4-b457-fdb1dd897007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:07.244725 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee787f-8946-4bb4-b457-fdb1dd897007 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:07.245005 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee787f-8946-4bb4-b457-fdb1dd897007 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:07.245280 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee787f-8946-4bb4-b457-fdb1dd897007 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:07.245952 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ee787f-8946-4bb4-b457-fdb1dd897007 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:07.246254 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ee787f-8946-4bb4-b457-fdb1dd897007 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:07.247048 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1970c63c-e13c-43f9-b6a6-0211d8d5d5bb None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:07.247608 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ee787f-8946-4bb4-b457-fdb1dd897007 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:07.265369 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 377/754] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:07.274877 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ee787f-8946-4bb4-b457-fdb1dd897007 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:07.306909 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53c3df6-30b8-4043-9cc8-a385d6902057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:07.307536 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c3df6-30b8-4043-9cc8-a385d6902057 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:07.307819 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c3df6-30b8-4043-9cc8-a385d6902057 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:07.308095 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c3df6-30b8-4043-9cc8-a385d6902057 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:07.308740 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53c3df6-30b8-4043-9cc8-a385d6902057 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:07.309044 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53c3df6-30b8-4043-9cc8-a385d6902057 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:07.310476 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53c3df6-30b8-4043-9cc8-a385d6902057 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:07.323666 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 378/755] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:07.325071 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8acbcc-2bd7-4791-9e12-562af0428dac None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:07.325365 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8acbcc-2bd7-4791-9e12-562af0428dac None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:07.325644 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8acbcc-2bd7-4791-9e12-562af0428dac None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:07.331139 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 379/756] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:07.332198 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8acbcc-2bd7-4791-9e12-562af0428dac None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:07.392525 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ee787f-8946-4bb4-b457-fdb1dd897007 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:07.424818 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:07.434423 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a6461-112b-41d0-a182-42e8a4e71493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:07.435055 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a6461-112b-41d0-a182-42e8a4e71493 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:07.435338 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a6461-112b-41d0-a182-42e8a4e71493 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:07.435610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a6461-112b-41d0-a182-42e8a4e71493 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:07.443667 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24a6461-112b-41d0-a182-42e8a4e71493 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:07.446973 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24a6461-112b-41d0-a182-42e8a4e71493 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:07.451574 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8acbcc-2bd7-4791-9e12-562af0428dac None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:07.459022 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-a24a6461-112b-41d0-a182-42e8a4e71493 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:07.459022 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:07.459022 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:07.459022 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:07.459022 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:07.459022 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:07.459022 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:07.459022 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:07.459022 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:07.459022 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:07.459022 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:07.459022 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:07.459022 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:29:07.459022 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:29:07.459022 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:29:07.459941 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:29:07.459941 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:07.459941 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:07.459941 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:07.459941 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:07.459941 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:07.459941 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:07.459941 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:07.459941 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:07.459941 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:07.459941 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:07.459941 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:07.459941 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:07.459941 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:07.459941 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:07.459941 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:07.459941 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:07.460902 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:29:07.460902 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:29:07.460902 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:29:07.460902 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:29:07.460902 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:29:07.460902 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:29:07.460902 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:07.460902 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:07.464895 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 378/757] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:29:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:07.465252 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:07.466393 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c543db-db8f-448e-9fd6-ddc4de90c2a4 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:07.466552 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c543db-db8f-448e-9fd6-ddc4de90c2a4 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:07.466781 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c543db-db8f-448e-9fd6-ddc4de90c2a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:07.486531 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-33c543db-db8f-448e-9fd6-ddc4de90c2a4 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:07.487712 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-33c543db-db8f-448e-9fd6-ddc4de90c2a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:07.498142 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211e8d7a-3052-47b5-a47c-a5b2f61cd788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:07.499502 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e8d7a-3052-47b5-a47c-a5b2f61cd788 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:07.500404 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e8d7a-3052-47b5-a47c-a5b2f61cd788 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:07.500904 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e8d7a-3052-47b5-a47c-a5b2f61cd788 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:07.510978 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211e8d7a-3052-47b5-a47c-a5b2f61cd788 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:07.513222 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211e8d7a-3052-47b5-a47c-a5b2f61cd788 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:07.529102 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-211e8d7a-3052-47b5-a47c-a5b2f61cd788 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 15:29:07.529102 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:07.529102 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:07.529102 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:07.529102 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:07.529102 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:07.529102 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:07.529102 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:07.529102 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:07.529102 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:07.529102 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:07.529102 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:07.529102 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:29:07.529102 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:29:07.529102 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:29:07.529102 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:29:07.530642 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:07.530642 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:07.530642 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:29:07.530642 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:07.530642 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:29:07.530642 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:07.530642 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:07.530642 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:07.530642 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:07.530642 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:07.530642 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:07.530642 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:07.530642 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:07.530642 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:07.530642 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:07.530642 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:07.530642 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:07.530642 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:07.531807 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:07.531807 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:07.531807 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:29:07.531807 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:29:07.531807 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:29:07.531807 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:29:07.531807 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:29:07.531807 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:29:07.531807 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 15:29:07.531807 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:07.534774 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 380/758] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Sep 28 15:29:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:07.535774 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211e8d7a-3052-47b5-a47c-a5b2f61cd788 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:07.572288 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 379/759] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:29:07.572288 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c543db-db8f-448e-9fd6-ddc4de90c2a4 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:07.711063 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211e8d7a-3052-47b5-a47c-a5b2f61cd788 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:07.743888 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c543db-db8f-448e-9fd6-ddc4de90c2a4 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:07.767722 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef39a1-18e9-4c04-b5a0-1bfde3bc1672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:07.768332 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef39a1-18e9-4c04-b5a0-1bfde3bc1672 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:07.768606 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef39a1-18e9-4c04-b5a0-1bfde3bc1672 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:07.768909 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef39a1-18e9-4c04-b5a0-1bfde3bc1672 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:07.769550 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ef39a1-18e9-4c04-b5a0-1bfde3bc1672 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:07.769868 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ef39a1-18e9-4c04-b5a0-1bfde3bc1672 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:07.771330 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ef39a1-18e9-4c04-b5a0-1bfde3bc1672 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:07.786149 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:07.798953 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 381/760] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 15:29:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:07.805723 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef39a1-18e9-4c04-b5a0-1bfde3bc1672 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:07.822074 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25267a9e-59b3-42a8-8c08-e541891343a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:07.822684 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25267a9e-59b3-42a8-8c08-e541891343a7 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:07.822954 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25267a9e-59b3-42a8-8c08-e541891343a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:07.823218 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25267a9e-59b3-42a8-8c08-e541891343a7 None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:07.823943 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25267a9e-59b3-42a8-8c08-e541891343a7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:07.824244 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25267a9e-59b3-42a8-8c08-e541891343a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:07.825732 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25267a9e-59b3-42a8-8c08-e541891343a7 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:07.857817 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 380/761] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Sep 28 15:29:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:07.861287 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:07.863315 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25267a9e-59b3-42a8-8c08-e541891343a7 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:07.968014 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef39a1-18e9-4c04-b5a0-1bfde3bc1672 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:07.978693 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25267a9e-59b3-42a8-8c08-e541891343a7 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:08.016492 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8accfaa5-d440-4ec8-ac29-823e4209d2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:08.017331 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8accfaa5-d440-4ec8-ac29-823e4209d2a7 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:08.017777 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8accfaa5-d440-4ec8-ac29-823e4209d2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:08.018235 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8accfaa5-d440-4ec8-ac29-823e4209d2a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:08.018917 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00610f7-5d0c-4ad8-a32c-b94898b83bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:08.019832 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00610f7-5d0c-4ad8-a32c-b94898b83bc5 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:08.020303 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00610f7-5d0c-4ad8-a32c-b94898b83bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:08.020753 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00610f7-5d0c-4ad8-a32c-b94898b83bc5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:08.022167 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00610f7-5d0c-4ad8-a32c-b94898b83bc5 None admin] RBAC: Adding query filter params (role.id=62d96ac9513c452894fc67b442cfc59f, scope.project.id=3dc6c5f5fcd34922b0d09cd2aaed6cf1, user.id=328051fe7607464f8f5dc0dcc01a817a) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:08.022689 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00610f7-5d0c-4ad8-a32c-b94898b83bc5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:08.026708 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00610f7-5d0c-4ad8-a32c-b94898b83bc5 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:08.041249 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 382/762] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 15:29:07 2022] GET /identity/v3/role_assignments?user.id=328051fe7607464f8f5dc0dcc01a817a&scope.project.id=3dc6c5f5fcd34922b0d09cd2aaed6cf1&role.id=62d96ac9513c452894fc67b442cfc59f => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:08.041249 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00610f7-5d0c-4ad8-a32c-b94898b83bc5 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:08.131557 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:08.178940 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8accfaa5-d440-4ec8-ac29-823e4209d2a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:08.181188 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8accfaa5-d440-4ec8-ac29-823e4209d2a7 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:08.187454 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:08.205256 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00610f7-5d0c-4ad8-a32c-b94898b83bc5 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:08.261361 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9106e131-77b1-406c-ad4c-64444518fade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:08.262289 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106e131-77b1-406c-ad4c-64444518fade None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:08.262719 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106e131-77b1-406c-ad4c-64444518fade None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:08.263140 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106e131-77b1-406c-ad4c-64444518fade None admin] PATH_INFO: `/v3/services/network` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:08.263876 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9106e131-77b1-406c-ad4c-64444518fade None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:08.265439 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9106e131-77b1-406c-ad4c-64444518fade None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:08.275779 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-9106e131-77b1-406c-ad4c-64444518fade None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 28 15:29:08.275779 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:08.275779 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:29:08.275779 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:29:08.275779 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:29:08.275779 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:29:08.275779 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:29:08.275779 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:29:08.275779 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 28 15:29:08.275779 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:08.275779 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:29:08.275779 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:08.275779 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:08.275779 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:08.275779 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:08.275779 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:08.275779 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:08.276806 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:08.276806 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:08.276806 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:08.276806 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:08.276806 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:08.276806 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:08.276806 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:29:08.276806 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:29:08.276806 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:29:08.276806 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:29:08.276806 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:08.276806 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:08.276806 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:08.276806 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:08.276806 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:08.276806 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:08.277630 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:08.277630 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:08.277630 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:08.277630 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:08.277630 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:08.277630 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:08.277630 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:08.277630 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:08.277630 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:08.277630 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:08.277630 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:29:08.277630 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:29:08.277630 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 28 15:29:08.277630 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:08.279922 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 383/763] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:29:08 2022] GET /identity/v3/services/network => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:08.285772 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9106e131-77b1-406c-ad4c-64444518fade None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:08.338690 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 381/764] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:29:07 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 15:29:08.344740 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8accfaa5-d440-4ec8-ac29-823e4209d2a7 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:08.404044 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9106e131-77b1-406c-ad4c-64444518fade None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:08.428140 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:08.442125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f24c010-218c-4721-a9a2-e686d94f707a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:08.442968 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f24c010-218c-4721-a9a2-e686d94f707a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:08.443398 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f24c010-218c-4721-a9a2-e686d94f707a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:08.443817 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f24c010-218c-4721-a9a2-e686d94f707a None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:08.444618 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f24c010-218c-4721-a9a2-e686d94f707a None admin] RBAC: Adding query filter params (name=network) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:08.445725 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f24c010-218c-4721-a9a2-e686d94f707a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:08.447151 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f24c010-218c-4721-a9a2-e686d94f707a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:08.452246 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 384/765] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Sep 28 15:29:08 2022] GET /identity/v3/services?name=network => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:08.453500 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8377ecc-ff4b-400a-ac96-4934da754bc9 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:08.453712 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8377ecc-ff4b-400a-ac96-4934da754bc9 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:08.458430 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8377ecc-ff4b-400a-ac96-4934da754bc9 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:08.459808 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8accfaa5-d440-4ec8-ac29-823e4209d2a7 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:08.464199 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 385/766] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:08.464199 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8377ecc-ff4b-400a-ac96-4934da754bc9 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:08.482705 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:08.504251 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5abd25-516d-4ffa-a0eb-d0c1eef00ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:08.505251 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5abd25-516d-4ffa-a0eb-d0c1eef00ad6 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:08.506022 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5abd25-516d-4ffa-a0eb-d0c1eef00ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:08.506549 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5abd25-516d-4ffa-a0eb-d0c1eef00ad6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:08.519460 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5abd25-516d-4ffa-a0eb-d0c1eef00ad6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:08.521507 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5abd25-516d-4ffa-a0eb-d0c1eef00ad6 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:08.537206 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-9d5abd25-516d-4ffa-a0eb-d0c1eef00ad6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:08.537206 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:08.537206 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:08.537206 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:08.537206 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:08.537206 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:08.537206 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:08.537206 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:08.537206 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:08.537206 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:08.537206 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:08.537206 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:08.537206 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:08.537206 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:08.537206 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:08.537206 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:08.538223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:08.538223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:08.538223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:08.538223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:08.538223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:08.538223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:08.538223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:08.538223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:08.538223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:08.538223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:08.538223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:08.538223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:08.538223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:08.538223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:08.538223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:08.538223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:08.538223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:08.539295 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:08.539295 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:08.539295 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:08.542437 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 382/767] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:08.543996 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8ac298-9df5-4ce8-9f87-10bb2e4f0f30 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:08.544553 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8ac298-9df5-4ce8-9f87-10bb2e4f0f30 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:08.545030 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8ac298-9df5-4ce8-9f87-10bb2e4f0f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:08.565906 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-7a8ac298-9df5-4ce8-9f87-10bb2e4f0f30 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:08.567419 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7a8ac298-9df5-4ce8-9f87-10bb2e4f0f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:08.569123 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8377ecc-ff4b-400a-ac96-4934da754bc9 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:08.604726 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0b172b-4173-470a-b50f-616b95896fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:08.605333 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0b172b-4173-470a-b50f-616b95896fc1 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:08.605655 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0b172b-4173-470a-b50f-616b95896fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:08.605956 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0b172b-4173-470a-b50f-616b95896fc1 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:08.606632 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0b172b-4173-470a-b50f-616b95896fc1 None admin] RBAC: Adding query filter params (type=network) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:08.606933 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0b172b-4173-470a-b50f-616b95896fc1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:08.608883 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0b172b-4173-470a-b50f-616b95896fc1 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:08.618788 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 386/768] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Sep 28 15:29:08 2022] GET /identity/v3/services?type=network => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:08.619680 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0b172b-4173-470a-b50f-616b95896fc1 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:08.649957 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 383/769] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:29:08.651257 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8ac298-9df5-4ce8-9f87-10bb2e4f0f30 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:08.706926 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:08.722656 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0b172b-4173-470a-b50f-616b95896fc1 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:08.761054 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7dcd57-8ab9-4466-8490-c5ef0cf0469b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:08.762314 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7dcd57-8ab9-4466-8490-c5ef0cf0469b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:08.762602 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7dcd57-8ab9-4466-8490-c5ef0cf0469b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:08.762875 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7dcd57-8ab9-4466-8490-c5ef0cf0469b None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:08.763513 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7dcd57-8ab9-4466-8490-c5ef0cf0469b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:08.763814 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7dcd57-8ab9-4466-8490-c5ef0cf0469b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:08.765200 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7dcd57-8ab9-4466-8490-c5ef0cf0469b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:08.768903 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:08.792470 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 387/770] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:08.797942 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7dcd57-8ab9-4466-8490-c5ef0cf0469b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:08.831237 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8ac298-9df5-4ce8-9f87-10bb2e4f0f30 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:08.899778 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4652394a-a175-456d-b9ba-1c35e5bc9bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:08.900397 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4652394a-a175-456d-b9ba-1c35e5bc9bb1 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:08.900674 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4652394a-a175-456d-b9ba-1c35e5bc9bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:08.900943 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4652394a-a175-456d-b9ba-1c35e5bc9bb1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:08.901720 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4652394a-a175-456d-b9ba-1c35e5bc9bb1 None admin] RBAC: Adding query filter params (interface=public, service_id=a58c8734b8c0405aa4b93a82aadeb3b3, region_id=RegionOne) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:08.902059 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4652394a-a175-456d-b9ba-1c35e5bc9bb1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:08.903430 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4652394a-a175-456d-b9ba-1c35e5bc9bb1 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:08.920861 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 384/771] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Sep 28 15:29:08 2022] GET /identity/v3/endpoints?service_id=a58c8734b8c0405aa4b93a82aadeb3b3&interface=public®ion_id=RegionOne => generated 194 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:08.926797 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4652394a-a175-456d-b9ba-1c35e5bc9bb1 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:08.975294 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7dcd57-8ab9-4466-8490-c5ef0cf0469b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:09.044457 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b847064-310e-4cd6-b987-3b489ba2c1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:09.045093 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b847064-310e-4cd6-b987-3b489ba2c1d7 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:09.045372 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b847064-310e-4cd6-b987-3b489ba2c1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:09.045641 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b847064-310e-4cd6-b987-3b489ba2c1d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:09.055019 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b847064-310e-4cd6-b987-3b489ba2c1d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:09.055797 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b847064-310e-4cd6-b987-3b489ba2c1d7 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:09.067814 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-6b847064-310e-4cd6-b987-3b489ba2c1d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:09.067814 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:09.067814 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:09.067814 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:09.067814 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:09.067814 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:09.067814 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:09.067814 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:09.067814 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:09.067814 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:09.067814 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:09.067814 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:09.067814 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:29:09.067814 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:29:09.067814 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:29:09.068721 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:29:09.068721 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:09.068721 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:09.068721 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:09.068721 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:09.068721 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:09.068721 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:09.068721 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:09.068721 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:09.068721 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:09.068721 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:09.068721 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:09.068721 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:09.068721 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:09.068721 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:09.068721 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:09.068721 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:09.070788 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:29:09.070788 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:29:09.070788 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:29:09.070788 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:29:09.070788 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:29:09.070788 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:29:09.070788 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:09.070788 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:09.070788 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 388/772] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:29:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:29:09.070788 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b847064-310e-4cd6-b987-3b489ba2c1d7 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:09.075777 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:09.098369 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4652394a-a175-456d-b9ba-1c35e5bc9bb1 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:09.133544 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:09.146085 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d0e67a-8f4f-4f38-bd00-55b6e8692ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:09.147248 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0e67a-8f4f-4f38-bd00-55b6e8692ec3 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:09.148267 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0e67a-8f4f-4f38-bd00-55b6e8692ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:09.148814 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0e67a-8f4f-4f38-bd00-55b6e8692ec3 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:09.149786 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d0e67a-8f4f-4f38-bd00-55b6e8692ec3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:09.152207 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d0e67a-8f4f-4f38-bd00-55b6e8692ec3 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:09.159045 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 385/773] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Sep 28 15:29:08 2022] GET /identity/v3/services => generated 772 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:09.160569 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d0e67a-8f4f-4f38-bd00-55b6e8692ec3 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:09.200740 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b847064-310e-4cd6-b987-3b489ba2c1d7 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:09.248404 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e08d782-c02d-4932-b55f-ad60e676376c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:09.249458 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e08d782-c02d-4932-b55f-ad60e676376c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:09.250003 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e08d782-c02d-4932-b55f-ad60e676376c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:09.250489 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e08d782-c02d-4932-b55f-ad60e676376c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:09.251292 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e08d782-c02d-4932-b55f-ad60e676376c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:09.252911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e08d782-c02d-4932-b55f-ad60e676376c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:09.265960 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-2e08d782-c02d-4932-b55f-ad60e676376c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 28 15:29:09.265960 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:09.265960 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:29:09.265960 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:29:09.265960 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:29:09.265960 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:29:09.265960 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:29:09.265960 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:29:09.265960 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 28 15:29:09.265960 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:09.265960 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:29:09.265960 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:09.265960 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:09.265960 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:09.265960 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:09.265960 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:09.265960 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:09.267039 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:09.267039 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:09.267039 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:09.267039 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:09.267039 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:09.267039 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:09.267039 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:29:09.267039 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:29:09.267039 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:29:09.267039 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:29:09.267039 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:09.267039 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:09.267039 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:09.267039 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:09.267039 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:09.267039 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:09.267908 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:09.267908 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:09.267908 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:09.267908 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:09.267908 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:09.267908 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:09.267908 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:09.267908 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:09.267908 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:09.267908 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:09.267908 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:29:09.267908 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:29:09.267908 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 28 15:29:09.267908 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:09.277596 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 389/774] 10.222.0.52 () {56 vars in 1129 bytes} [Wed Sep 28 15:29:09 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:09.279287 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e08d782-c02d-4932-b55f-ad60e676376c None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:09.370913 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:09.394141 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d0e67a-8f4f-4f38-bd00-55b6e8692ec3 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:09.418153 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e08d782-c02d-4932-b55f-ad60e676376c None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:09.434321 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e973e78-65cd-4392-bdc9-13f752471f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:09.435587 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e973e78-65cd-4392-bdc9-13f752471f51 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:09.436103 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e973e78-65cd-4392-bdc9-13f752471f51 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:09.436590 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e973e78-65cd-4392-bdc9-13f752471f51 None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:09.437484 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e973e78-65cd-4392-bdc9-13f752471f51 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:09.438045 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e973e78-65cd-4392-bdc9-13f752471f51 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:09.439684 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e973e78-65cd-4392-bdc9-13f752471f51 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:09.453975 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 386/775] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 15:29:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:09.455211 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f125dcc-7133-4f15-b0e4-d3f05e9d41a9 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:09.455489 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f125dcc-7133-4f15-b0e4-d3f05e9d41a9 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:09.455782 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f125dcc-7133-4f15-b0e4-d3f05e9d41a9 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:09.457138 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 387/776] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:09.458698 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a72946-32b1-4275-85da-e26cb3ac1cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:09.459383 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a72946-32b1-4275-85da-e26cb3ac1cdb None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:09.459671 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a72946-32b1-4275-85da-e26cb3ac1cdb None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:09.459945 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a72946-32b1-4275-85da-e26cb3ac1cdb None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:09.461233 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a72946-32b1-4275-85da-e26cb3ac1cdb None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:09.461539 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a72946-32b1-4275-85da-e26cb3ac1cdb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:09.463050 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a72946-32b1-4275-85da-e26cb3ac1cdb None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:09.464890 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac7301-6371-4718-af0a-13ea71f18353 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:09.465168 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac7301-6371-4718-af0a-13ea71f18353 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:09.465438 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac7301-6371-4718-af0a-13ea71f18353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:09.468366 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 390/777] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Sep 28 15:29:09 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:09.469959 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a72946-32b1-4275-85da-e26cb3ac1cdb None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:09.488504 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-b7ac7301-6371-4718-af0a-13ea71f18353 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:09.490698 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b7ac7301-6371-4718-af0a-13ea71f18353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:09.544407 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:09.567295 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 388/778] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:29:09.574884 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ac7301-6371-4718-af0a-13ea71f18353 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:09.604165 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a72946-32b1-4275-85da-e26cb3ac1cdb None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:09.646319 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:09.650491 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ca458-1e04-42b7-b4a2-9830a6e74a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:09.651120 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ca458-1e04-42b7-b4a2-9830a6e74a53 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:09.654470 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ca458-1e04-42b7-b4a2-9830a6e74a53 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:09.654470 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ca458-1e04-42b7-b4a2-9830a6e74a53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:09.654470 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97ca458-1e04-42b7-b4a2-9830a6e74a53 None admin] RBAC: Adding query filter params (role.id=342b64fe0cdb4925b23c6cd461a152ac, scope.project.id=3dc6c5f5fcd34922b0d09cd2aaed6cf1, user.id=c81e5fb007614cf2992f5e02e85d6307) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:09.654470 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97ca458-1e04-42b7-b4a2-9830a6e74a53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:09.654470 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97ca458-1e04-42b7-b4a2-9830a6e74a53 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:09.662814 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 391/779] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 15:29:09 2022] GET /identity/v3/role_assignments?user.id=c81e5fb007614cf2992f5e02e85d6307&scope.project.id=3dc6c5f5fcd34922b0d09cd2aaed6cf1&role.id=342b64fe0cdb4925b23c6cd461a152ac => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:09.796028 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ac7301-6371-4718-af0a-13ea71f18353 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:09.847682 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2222727-37c5-4ed0-ace5-7779b42b3343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:09.848294 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2222727-37c5-4ed0-ace5-7779b42b3343 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:09.848565 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2222727-37c5-4ed0-ace5-7779b42b3343 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:09.848833 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2222727-37c5-4ed0-ace5-7779b42b3343 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:09.849456 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2222727-37c5-4ed0-ace5-7779b42b3343 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:09.849749 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2222727-37c5-4ed0-ace5-7779b42b3343 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:09.851789 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2222727-37c5-4ed0-ace5-7779b42b3343 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:09.863245 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 389/780] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Sep 28 15:29:09 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:09.869595 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ca458-1e04-42b7-b4a2-9830a6e74a53 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:09.963689 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2222727-37c5-4ed0-ace5-7779b42b3343 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:10.030280 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:10.036668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ca458-1e04-42b7-b4a2-9830a6e74a53 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:10.092721 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2222727-37c5-4ed0-ace5-7779b42b3343 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:10.127992 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671dcfa8-0e51-413c-bc34-feb20b366269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:10.128643 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671dcfa8-0e51-413c-bc34-feb20b366269 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:10.128921 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671dcfa8-0e51-413c-bc34-feb20b366269 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:10.129194 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671dcfa8-0e51-413c-bc34-feb20b366269 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:10.129913 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671dcfa8-0e51-413c-bc34-feb20b366269 None admin] RBAC: Adding query filter params (interface=public, service_id=0bb7abfd857648e7812375efedd982df, region_id=RegionOne) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:10.130209 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671dcfa8-0e51-413c-bc34-feb20b366269 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:10.131838 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671dcfa8-0e51-413c-bc34-feb20b366269 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:10.144564 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b44be7-6e84-4a8a-bb1d-fa655d2081c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:10.145530 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b44be7-6e84-4a8a-bb1d-fa655d2081c6 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:10.146063 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b44be7-6e84-4a8a-bb1d-fa655d2081c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:10.146549 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b44be7-6e84-4a8a-bb1d-fa655d2081c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:10.160488 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 392/781] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Sep 28 15:29:09 2022] GET /identity/v3/endpoints?service_id=0bb7abfd857648e7812375efedd982df&interface=public®ion_id=RegionOne => generated 194 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:10.161782 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671dcfa8-0e51-413c-bc34-feb20b366269 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:10.178764 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b44be7-6e84-4a8a-bb1d-fa655d2081c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:10.185647 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-a0b44be7-6e84-4a8a-bb1d-fa655d2081c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:29:10.185647 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:10.185647 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:10.185647 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:10.185647 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:10.185647 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:10.185647 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:10.185647 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:10.185647 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:10.185647 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:10.185647 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:10.185647 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:10.185647 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:10.185647 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:10.185647 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:10.185647 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:10.186626 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:10.186626 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:10.186626 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:10.186626 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:10.186626 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:10.186626 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:10.186626 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:10.186626 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:10.186626 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:10.186626 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:10.186626 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:10.186626 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:10.186626 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:10.186626 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:10.186626 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:10.186626 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:10.187459 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:10.187459 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:10.187459 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:10.187459 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:10.187459 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:10.187459 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:10.187459 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:10.187459 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:10.187459 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:10.187459 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:10.187459 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:29:10.187459 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:10.191601 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 390/782] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 15:29:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:29:10.197869 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b44be7-6e84-4a8a-bb1d-fa655d2081c6 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:10.305462 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:10.386886 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b44be7-6e84-4a8a-bb1d-fa655d2081c6 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:10.390190 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:10.400604 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671dcfa8-0e51-413c-bc34-feb20b366269 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:10.446724 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed0c5fd-dc9f-472c-9e08-820d73217bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:10.447693 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed0c5fd-dc9f-472c-9e08-820d73217bfc None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:10.447975 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed0c5fd-dc9f-472c-9e08-820d73217bfc None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:10.448249 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed0c5fd-dc9f-472c-9e08-820d73217bfc None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:10.448911 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed0c5fd-dc9f-472c-9e08-820d73217bfc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:10.449213 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed0c5fd-dc9f-472c-9e08-820d73217bfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:10.450883 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed0c5fd-dc9f-472c-9e08-820d73217bfc None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:10.460126 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 391/783] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:29:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:10.466279 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed0c5fd-dc9f-472c-9e08-820d73217bfc None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:10.486620 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63904bcd-e322-4f3a-9b63-b1f0818d3163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:10.487244 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63904bcd-e322-4f3a-9b63-b1f0818d3163 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:10.487569 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63904bcd-e322-4f3a-9b63-b1f0818d3163 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:10.487845 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63904bcd-e322-4f3a-9b63-b1f0818d3163 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:10.488453 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63904bcd-e322-4f3a-9b63-b1f0818d3163 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:10.489811 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63904bcd-e322-4f3a-9b63-b1f0818d3163 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:10.510242 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 393/784] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Sep 28 15:29:10 2022] GET /identity/v3/services => generated 772 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:10.580416 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed0c5fd-dc9f-472c-9e08-820d73217bfc None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:10.616765 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998c7d62-d3df-4b23-a2ea-ecbf27a30414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:10.617487 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c7d62-d3df-4b23-a2ea-ecbf27a30414 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:10.617786 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c7d62-d3df-4b23-a2ea-ecbf27a30414 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:10.618077 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c7d62-d3df-4b23-a2ea-ecbf27a30414 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:10.630064 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998c7d62-d3df-4b23-a2ea-ecbf27a30414 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:10.631784 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998c7d62-d3df-4b23-a2ea-ecbf27a30414 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:10.647522 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-998c7d62-d3df-4b23-a2ea-ecbf27a30414 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:10.647522 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:10.647522 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:10.647522 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:10.647522 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:10.647522 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:10.647522 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:10.647522 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:10.647522 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:10.647522 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:10.647522 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:10.647522 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:10.647522 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:10.647522 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:10.647522 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:10.647522 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:10.649072 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:10.649072 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:10.649072 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:10.649072 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:10.649072 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:10.649072 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:10.649072 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:10.649072 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:10.649072 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:10.649072 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:10.649072 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:10.649072 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:10.649072 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:10.649072 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:10.649072 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:10.649072 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:10.649072 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:10.651080 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:10.651080 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:10.651080 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:10.651080 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 392/785] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:10.655026 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63904bcd-e322-4f3a-9b63-b1f0818d3163 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:10.657563 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c08e6-18fa-4ce9-b45f-4fbf7ac036a1 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:10.657836 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c08e6-18fa-4ce9-b45f-4fbf7ac036a1 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:10.658232 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c08e6-18fa-4ce9-b45f-4fbf7ac036a1 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:10.661090 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 393/786] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:10.666152 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fbd540-8d34-4851-8937-deba400acac6 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:10.666440 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fbd540-8d34-4851-8937-deba400acac6 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:10.666662 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fbd540-8d34-4851-8937-deba400acac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:10.690967 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-c8fbd540-8d34-4851-8937-deba400acac6 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:10.692194 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c8fbd540-8d34-4851-8937-deba400acac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:10.722213 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:10.772578 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63904bcd-e322-4f3a-9b63-b1f0818d3163 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:10.798019 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:10.814412 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1664b4-a68e-46bd-a5aa-87ad101b16c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:10.815073 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1664b4-a68e-46bd-a5aa-87ad101b16c2 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:10.815352 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1664b4-a68e-46bd-a5aa-87ad101b16c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:10.815671 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1664b4-a68e-46bd-a5aa-87ad101b16c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:10.816306 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1664b4-a68e-46bd-a5aa-87ad101b16c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:10.816602 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1664b4-a68e-46bd-a5aa-87ad101b16c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:10.818001 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1664b4-a68e-46bd-a5aa-87ad101b16c2 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:10.831145 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 394/787] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 15:29:10.831309 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 394/788] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:10.837057 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fbd540-8d34-4851-8937-deba400acac6 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:10.942606 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fbd540-8d34-4851-8937-deba400acac6 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:10.977288 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faddb0dd-e7fd-468c-adc4-e4a9ead18c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:10.977932 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faddb0dd-e7fd-468c-adc4-e4a9ead18c52 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:10.978248 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faddb0dd-e7fd-468c-adc4-e4a9ead18c52 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:10.978474 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faddb0dd-e7fd-468c-adc4-e4a9ead18c52 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:10.985969 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faddb0dd-e7fd-468c-adc4-e4a9ead18c52 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:10.987396 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faddb0dd-e7fd-468c-adc4-e4a9ead18c52 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:10.999448 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-faddb0dd-e7fd-468c-adc4-e4a9ead18c52 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 28 15:29:10.999448 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:10.999448 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:10.999448 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:10.999448 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:10.999448 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:10.999448 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:10.999448 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:10.999448 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:10.999448 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:10.999448 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:10.999448 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:10.999448 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:29:10.999448 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:29:10.999448 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:29:10.999448 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:29:11.000423 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:11.000423 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:11.000423 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:29:11.000423 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:11.000423 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:29:11.000423 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:11.000423 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:11.000423 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:11.000423 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:11.000423 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:11.000423 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:11.000423 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:11.000423 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:11.000423 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:11.000423 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:11.000423 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:11.000423 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:11.000423 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:11.002419 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:11.002419 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:11.002419 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:29:11.002419 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:29:11.002419 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:29:11.002419 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:29:11.002419 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:29:11.002419 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:29:11.002419 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 28 15:29:11.002419 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:11.002419 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 395/789] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Sep 28 15:29:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:11.006603 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1664b4-a68e-46bd-a5aa-87ad101b16c2 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:11.132995 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faddb0dd-e7fd-468c-adc4-e4a9ead18c52 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:11.145421 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:11.155887 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1664b4-a68e-46bd-a5aa-87ad101b16c2 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:11.225786 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b1f54-29fe-41ac-a556-f9fd94a7104c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:11.226939 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b1f54-29fe-41ac-a556-f9fd94a7104c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:11.227221 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b1f54-29fe-41ac-a556-f9fd94a7104c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:11.227491 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b1f54-29fe-41ac-a556-f9fd94a7104c None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:11.228252 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053b1f54-29fe-41ac-a556-f9fd94a7104c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:11.228603 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053b1f54-29fe-41ac-a556-f9fd94a7104c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:11.230072 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053b1f54-29fe-41ac-a556-f9fd94a7104c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:11.251035 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 395/790] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Sep 28 15:29:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:11.257314 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b1f54-29fe-41ac-a556-f9fd94a7104c None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:11.318333 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faddb0dd-e7fd-468c-adc4-e4a9ead18c52 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:11.343209 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:11.396644 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c15516-2241-4ae6-af47-7269b7b7495f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:11.397190 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c15516-2241-4ae6-af47-7269b7b7495f None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:11.397468 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c15516-2241-4ae6-af47-7269b7b7495f None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:11.397789 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c15516-2241-4ae6-af47-7269b7b7495f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:11.416164 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c15516-2241-4ae6-af47-7269b7b7495f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:11.425252 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-26c15516-2241-4ae6-af47-7269b7b7495f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:29:11.425252 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:11.425252 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:11.425252 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:11.425252 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:11.425252 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:11.425252 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:11.425252 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:11.425252 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:11.425252 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:11.425252 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:11.425252 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:11.425252 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:11.425252 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:11.425252 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:11.425252 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:11.426237 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:11.426237 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:11.426237 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:11.426237 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:11.426237 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:11.426237 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:11.426237 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:11.426237 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:11.426237 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:11.426237 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:11.426237 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:11.426237 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:11.426237 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:11.426237 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:11.426237 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:11.426237 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:11.428413 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:11.428413 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:11.428413 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:11.428413 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:11.428413 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:11.428413 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:11.428413 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:11.428413 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:11.428413 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:11.428413 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:11.428413 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:29:11.428413 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:11.428413 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 396/791] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 15:29:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:11.434998 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c15516-2241-4ae6-af47-7269b7b7495f None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:11.444012 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b1f54-29fe-41ac-a556-f9fd94a7104c None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:11.510703 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e5f12-0ae1-4b62-9c46-e93922911503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:11.511702 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e5f12-0ae1-4b62-9c46-e93922911503 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:11.512191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e5f12-0ae1-4b62-9c46-e93922911503 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:11.512694 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e5f12-0ae1-4b62-9c46-e93922911503 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:11.576935 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:11.614556 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590e5f12-0ae1-4b62-9c46-e93922911503 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:11.616473 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590e5f12-0ae1-4b62-9c46-e93922911503 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:11.671555 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c15516-2241-4ae6-af47-7269b7b7495f None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:11.686197 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:11.730071 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868ef7d8-cad3-48f1-b06e-31c77a9ac2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:11.730684 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ef7d8-cad3-48f1-b06e-31c77a9ac2a8 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:11.733364 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ef7d8-cad3-48f1-b06e-31c77a9ac2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:11.733655 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ef7d8-cad3-48f1-b06e-31c77a9ac2a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:11.734357 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868ef7d8-cad3-48f1-b06e-31c77a9ac2a8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:11.734664 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868ef7d8-cad3-48f1-b06e-31c77a9ac2a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:11.736024 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868ef7d8-cad3-48f1-b06e-31c77a9ac2a8 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:11.748030 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 397/792] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:29:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:11.749277 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ee699-4d8e-4522-986b-ec95bbe98237 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:11.749560 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ee699-4d8e-4522-986b-ec95bbe98237 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:11.749923 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ee699-4d8e-4522-986b-ec95bbe98237 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:11.752360 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 398/793] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:11.755172 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884ee699-4d8e-4522-986b-ec95bbe98237 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:11.782420 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 396/794] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:29:11 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 15:29:11.784674 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d06a3-de73-4b3a-948a-616d5ca2acb9 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:11.785355 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d06a3-de73-4b3a-948a-616d5ca2acb9 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:11.785802 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573d06a3-de73-4b3a-948a-616d5ca2acb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:11.808341 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-573d06a3-de73-4b3a-948a-616d5ca2acb9 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:11.810139 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-573d06a3-de73-4b3a-948a-616d5ca2acb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:11.871420 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884ee699-4d8e-4522-986b-ec95bbe98237 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:11.885874 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 397/795] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:29:11.885874 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573d06a3-de73-4b3a-948a-616d5ca2acb9 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:11.926345 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:11.938240 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3143c0bb-e14d-4476-a2f6-1aa1f880726b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:11.939590 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3143c0bb-e14d-4476-a2f6-1aa1f880726b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:11.939590 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3143c0bb-e14d-4476-a2f6-1aa1f880726b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:11.939590 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3143c0bb-e14d-4476-a2f6-1aa1f880726b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:11.958329 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:11.969874 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3143c0bb-e14d-4476-a2f6-1aa1f880726b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:11.969874 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3143c0bb-e14d-4476-a2f6-1aa1f880726b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:11.993229 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-3143c0bb-e14d-4476-a2f6-1aa1f880726b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:11.993229 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:11.993229 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:11.993229 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:11.993229 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:11.993229 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:11.993229 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:11.993229 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:11.993229 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:11.993229 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:11.993229 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:11.993229 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:11.993229 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:11.993229 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:11.993229 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:11.993229 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:11.994242 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:11.994242 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:11.994242 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:11.994242 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:11.994242 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:11.994242 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:11.994242 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:11.994242 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:11.994242 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:11.994242 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:11.994242 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:11.994242 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:11.994242 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:11.994242 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:11.994242 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:11.994242 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:11.994242 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:11.995108 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:11.995108 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:11.995108 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:11.995108 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 399/796] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:29:12.002267 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3143c0bb-e14d-4476-a2f6-1aa1f880726b None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:12.032212 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573d06a3-de73-4b3a-948a-616d5ca2acb9 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:12.101872 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be6b3b5-7522-4815-b5fd-12aa90c5c657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:12.103584 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be6b3b5-7522-4815-b5fd-12aa90c5c657 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:12.104715 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be6b3b5-7522-4815-b5fd-12aa90c5c657 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:12.105194 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be6b3b5-7522-4815-b5fd-12aa90c5c657 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:12.124944 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be6b3b5-7522-4815-b5fd-12aa90c5c657 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:12.131830 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be6b3b5-7522-4815-b5fd-12aa90c5c657 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:12.156971 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-6be6b3b5-7522-4815-b5fd-12aa90c5c657 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:12.156971 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:12.156971 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:12.156971 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:12.156971 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:12.156971 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:12.156971 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:12.156971 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:12.156971 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:12.156971 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:12.156971 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:12.156971 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:12.156971 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:12.156971 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:12.156971 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:12.156971 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:12.157967 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:12.157967 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:12.157967 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:12.157967 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:12.157967 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:12.157967 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:12.157967 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:12.157967 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:12.157967 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:12.157967 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:12.157967 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:12.157967 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:12.157967 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:12.157967 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:12.157967 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:12.157967 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:12.157967 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:12.158857 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:12.158857 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:12.158857 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:12.163363 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 398/797] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:12.168772 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be6b3b5-7522-4815-b5fd-12aa90c5c657 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:12.186205 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3143c0bb-e14d-4476-a2f6-1aa1f880726b None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:12.254606 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4db0a96-f31f-4d94-b553-c04763629cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:12.255214 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db0a96-f31f-4d94-b553-c04763629cf4 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:12.255746 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db0a96-f31f-4d94-b553-c04763629cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:12.256015 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db0a96-f31f-4d94-b553-c04763629cf4 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:12.256785 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4db0a96-f31f-4d94-b553-c04763629cf4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:12.257002 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4db0a96-f31f-4d94-b553-c04763629cf4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:12.258903 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4db0a96-f31f-4d94-b553-c04763629cf4 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:12.271012 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 400/798] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:12.271837 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4db0a96-f31f-4d94-b553-c04763629cf4 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:12.317512 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be6b3b5-7522-4815-b5fd-12aa90c5c657 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:12.318340 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:12.343859 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:12.356530 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee29347-63d6-4ccb-8359-9e9d3abee3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:12.357135 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee29347-63d6-4ccb-8359-9e9d3abee3e9 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:12.357421 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee29347-63d6-4ccb-8359-9e9d3abee3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:12.357697 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee29347-63d6-4ccb-8359-9e9d3abee3e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:12.358363 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee29347-63d6-4ccb-8359-9e9d3abee3e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:12.358665 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee29347-63d6-4ccb-8359-9e9d3abee3e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:12.360030 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee29347-63d6-4ccb-8359-9e9d3abee3e9 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:12.372256 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 399/799] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:12.373037 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee29347-63d6-4ccb-8359-9e9d3abee3e9 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:12.381749 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4db0a96-f31f-4d94-b553-c04763629cf4 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:12.419202 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b1043a-426e-4a07-ae59-664020068159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:12.419824 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b1043a-426e-4a07-ae59-664020068159 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:12.420097 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b1043a-426e-4a07-ae59-664020068159 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:12.420361 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b1043a-426e-4a07-ae59-664020068159 None admin] PATH_INFO: `/v3/services/network` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:12.420931 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b1043a-426e-4a07-ae59-664020068159 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:12.422314 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b1043a-426e-4a07-ae59-664020068159 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:12.431714 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-d9b1043a-426e-4a07-ae59-664020068159 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 28 15:29:12.431714 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:12.431714 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:29:12.431714 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:29:12.431714 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:29:12.431714 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:29:12.431714 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:29:12.431714 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:29:12.431714 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 28 15:29:12.431714 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:12.431714 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:29:12.431714 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:12.431714 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:12.431714 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:12.431714 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:12.431714 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:12.431714 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:12.432708 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:12.432708 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:12.432708 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:12.432708 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:12.432708 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:12.432708 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:12.432708 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:29:12.432708 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:29:12.432708 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:29:12.432708 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:29:12.432708 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:12.432708 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:12.432708 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:12.432708 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:12.432708 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:12.432708 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:12.433574 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:12.433574 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:12.433574 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:12.433574 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:12.433574 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:12.433574 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:12.433574 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:12.433574 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:12.433574 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:12.433574 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:12.433574 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:29:12.433574 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:29:12.433574 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 28 15:29:12.433574 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:12.433574 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 401/800] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:29:12 2022] GET /identity/v3/services/network => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:12.435069 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b74feb-8d73-4c16-ba83-6088d618fe35 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:12.435069 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b74feb-8d73-4c16-ba83-6088d618fe35 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:12.435069 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b74feb-8d73-4c16-ba83-6088d618fe35 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:12.437976 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 402/801] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:12.437976 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b74feb-8d73-4c16-ba83-6088d618fe35 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:12.485704 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee29347-63d6-4ccb-8359-9e9d3abee3e9 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:12.521067 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab34596b-42be-445c-8b7e-e4afc8ad2d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:12.521693 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34596b-42be-445c-8b7e-e4afc8ad2d8a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:12.522009 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34596b-42be-445c-8b7e-e4afc8ad2d8a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:12.522284 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34596b-42be-445c-8b7e-e4afc8ad2d8a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:12.529704 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab34596b-42be-445c-8b7e-e4afc8ad2d8a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:12.531360 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab34596b-42be-445c-8b7e-e4afc8ad2d8a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:12.543527 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-ab34596b-42be-445c-8b7e-e4afc8ad2d8a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 28 15:29:12.543527 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:12.543527 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:12.543527 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:12.543527 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:12.543527 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:12.543527 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:12.543527 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:12.543527 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:12.543527 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:12.543527 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:12.543527 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:12.543527 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:29:12.543527 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:29:12.543527 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:29:12.543527 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:29:12.544457 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:12.544457 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:12.544457 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:29:12.544457 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:12.544457 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:29:12.544457 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:12.544457 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:12.544457 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:12.544457 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:12.544457 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:12.544457 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:12.544457 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:12.544457 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:12.544457 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:12.544457 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:12.544457 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:12.544457 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:12.544457 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:12.545379 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:12.545379 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:12.545379 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:29:12.545379 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:29:12.545379 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:29:12.545379 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:29:12.545379 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:29:12.545379 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:29:12.545379 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 28 15:29:12.545379 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:12.545379 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 400/802] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 15:29:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:12.545379 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab34596b-42be-445c-8b7e-e4afc8ad2d8a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:12.549884 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b74feb-8d73-4c16-ba83-6088d618fe35 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:12.559432 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:12.588429 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef779b44-4d04-4e5e-9773-c487a8448baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:12.589106 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef779b44-4d04-4e5e-9773-c487a8448baf None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:12.589341 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef779b44-4d04-4e5e-9773-c487a8448baf None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:12.589832 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef779b44-4d04-4e5e-9773-c487a8448baf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:12.593727 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:12.596837 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef779b44-4d04-4e5e-9773-c487a8448baf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:12.598347 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef779b44-4d04-4e5e-9773-c487a8448baf None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:12.610447 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-ef779b44-4d04-4e5e-9773-c487a8448baf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:12.610447 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:12.610447 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:12.610447 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:12.610447 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:12.610447 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:12.610447 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:12.610447 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:12.610447 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:12.610447 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:12.610447 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:12.610447 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:12.610447 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:29:12.610447 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:29:12.610447 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:29:12.611371 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:29:12.611371 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:12.611371 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:12.611371 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:12.611371 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:12.611371 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:12.611371 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:12.611371 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:12.611371 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:12.611371 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:12.611371 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:12.611371 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:12.611371 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:12.611371 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:12.611371 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:12.611371 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:12.611371 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:12.612240 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:29:12.612240 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:29:12.612240 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:29:12.612240 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:29:12.612240 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:29:12.612240 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:29:12.612240 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:12.612240 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:12.612240 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 403/803] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:29:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:12.612716 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17196a5e-9795-4e2c-994b-5f4b55f66027 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:12.613378 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17196a5e-9795-4e2c-994b-5f4b55f66027 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:12.613378 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17196a5e-9795-4e2c-994b-5f4b55f66027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:12.631977 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-17196a5e-9795-4e2c-994b-5f4b55f66027 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:12.633167 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-17196a5e-9795-4e2c-994b-5f4b55f66027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:12.658006 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab34596b-42be-445c-8b7e-e4afc8ad2d8a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:12.694454 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ab682f-183c-43bd-86a0-29c9fefb2935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:12.695068 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ab682f-183c-43bd-86a0-29c9fefb2935 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:12.695287 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ab682f-183c-43bd-86a0-29c9fefb2935 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:12.695559 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ab682f-183c-43bd-86a0-29c9fefb2935 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:12.696175 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ab682f-183c-43bd-86a0-29c9fefb2935 None admin] RBAC: Adding query filter params (name=network) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:12.696641 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ab682f-183c-43bd-86a0-29c9fefb2935 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:12.697859 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ab682f-183c-43bd-86a0-29c9fefb2935 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:12.702745 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 401/804] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Sep 28 15:29:12 2022] GET /identity/v3/services?name=network => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:12.704130 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 404/805] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:29:12.707795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ab682f-183c-43bd-86a0-29c9fefb2935 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:12.714883 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17196a5e-9795-4e2c-994b-5f4b55f66027 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:12.827166 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ab682f-183c-43bd-86a0-29c9fefb2935 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:12.837883 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:12.841698 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:12.844267 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17196a5e-9795-4e2c-994b-5f4b55f66027 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:12.872864 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99a766a-3412-4a7c-bb5c-65d9b67ef612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:12.873609 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a766a-3412-4a7c-bb5c-65d9b67ef612 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:12.874040 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a766a-3412-4a7c-bb5c-65d9b67ef612 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:12.874419 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a766a-3412-4a7c-bb5c-65d9b67ef612 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:12.875276 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99a766a-3412-4a7c-bb5c-65d9b67ef612 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:12.875656 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99a766a-3412-4a7c-bb5c-65d9b67ef612 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:12.877204 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99a766a-3412-4a7c-bb5c-65d9b67ef612 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:12.890363 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121009fd-4545-43c2-8475-1b43ea139751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:12.891203 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121009fd-4545-43c2-8475-1b43ea139751 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:12.891657 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121009fd-4545-43c2-8475-1b43ea139751 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:12.892700 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121009fd-4545-43c2-8475-1b43ea139751 None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:12.893568 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121009fd-4545-43c2-8475-1b43ea139751 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:12.894321 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121009fd-4545-43c2-8475-1b43ea139751 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:12.895911 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121009fd-4545-43c2-8475-1b43ea139751 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:12.901182 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 402/806] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Sep 28 15:29:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:12.902171 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99a766a-3412-4a7c-bb5c-65d9b67ef612 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:12.912047 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 405/807] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 15:29:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:12.913086 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121009fd-4545-43c2-8475-1b43ea139751 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:13.029202 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99a766a-3412-4a7c-bb5c-65d9b67ef612 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:13.036321 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121009fd-4545-43c2-8475-1b43ea139751 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:13.065579 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3213281-5d11-4bf5-b04a-51353920b27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:13.066217 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3213281-5d11-4bf5-b04a-51353920b27c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:13.066497 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3213281-5d11-4bf5-b04a-51353920b27c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:13.066768 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3213281-5d11-4bf5-b04a-51353920b27c None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:13.067386 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3213281-5d11-4bf5-b04a-51353920b27c None admin] RBAC: Adding query filter params (type=network) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:13.067685 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3213281-5d11-4bf5-b04a-51353920b27c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:13.069159 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3213281-5d11-4bf5-b04a-51353920b27c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:13.072057 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a613e425-3bd1-4e2f-843a-1fa6278eb022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:13.072684 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a613e425-3bd1-4e2f-843a-1fa6278eb022 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:13.072964 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a613e425-3bd1-4e2f-843a-1fa6278eb022 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:13.073237 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a613e425-3bd1-4e2f-843a-1fa6278eb022 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:13.074583 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 403/808] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Sep 28 15:29:12 2022] GET /identity/v3/services?type=network => generated 350 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:13.075386 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3213281-5d11-4bf5-b04a-51353920b27c None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:13.091868 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a613e425-3bd1-4e2f-843a-1fa6278eb022 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:13.094188 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a613e425-3bd1-4e2f-843a-1fa6278eb022 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:13.106503 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:13.111992 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-a613e425-3bd1-4e2f-843a-1fa6278eb022 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:13.111992 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:13.111992 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:13.111992 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:13.111992 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:13.111992 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:13.111992 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:13.111992 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:13.111992 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:13.111992 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:13.111992 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:13.111992 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:13.111992 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:13.111992 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:13.111992 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:13.111992 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:13.113138 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:13.113138 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:13.113138 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:13.113138 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:13.113138 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:13.113138 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:13.113138 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:13.113138 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:13.113138 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:13.113138 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:13.113138 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:13.113138 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:13.113138 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:13.113138 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:13.113138 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:13.113138 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:13.113138 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:13.114253 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:13.114253 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:13.114253 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:13.114253 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 406/809] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:13.114253 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a613e425-3bd1-4e2f-843a-1fa6278eb022 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:13.124748 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:13.182878 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3213281-5d11-4bf5-b04a-51353920b27c None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:13.213333 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a613e425-3bd1-4e2f-843a-1fa6278eb022 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:13.216521 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e5bbc-4ad5-49b9-83ab-e0a425c21880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:13.217118 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e5bbc-4ad5-49b9-83ab-e0a425c21880 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:13.217397 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e5bbc-4ad5-49b9-83ab-e0a425c21880 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:13.217670 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e5bbc-4ad5-49b9-83ab-e0a425c21880 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:13.218412 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8e5bbc-4ad5-49b9-83ab-e0a425c21880 None admin] RBAC: Adding query filter params (role.id=62d96ac9513c452894fc67b442cfc59f, scope.project.id=3dc6c5f5fcd34922b0d09cd2aaed6cf1, user.id=8daf14b64bf04b2ba74d2fe1fd41ff71) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:13.218722 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8e5bbc-4ad5-49b9-83ab-e0a425c21880 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:13.220163 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8e5bbc-4ad5-49b9-83ab-e0a425c21880 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:13.225782 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 404/810] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 15:29:13 2022] GET /identity/v3/role_assignments?user.id=8daf14b64bf04b2ba74d2fe1fd41ff71&scope.project.id=3dc6c5f5fcd34922b0d09cd2aaed6cf1&role.id=62d96ac9513c452894fc67b442cfc59f => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:13.226551 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e5bbc-4ad5-49b9-83ab-e0a425c21880 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:13.247445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60119bc-7726-4eee-9023-6509d7136c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:13.248099 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60119bc-7726-4eee-9023-6509d7136c6b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:13.248373 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60119bc-7726-4eee-9023-6509d7136c6b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:13.248640 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60119bc-7726-4eee-9023-6509d7136c6b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:13.260963 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60119bc-7726-4eee-9023-6509d7136c6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:13.266414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-c60119bc-7726-4eee-9023-6509d7136c6b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:29:13.266414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:13.266414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:13.266414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:13.266414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:13.266414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:13.266414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:13.266414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:13.266414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:13.266414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:13.266414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:13.266414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:13.266414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:13.266414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:13.266414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:13.266414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:13.267618 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:13.267618 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:13.267618 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:13.267618 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:13.267618 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:13.267618 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:13.267618 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:13.267618 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:13.267618 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:13.267618 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:13.267618 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:13.267618 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:13.267618 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:13.267618 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:13.267618 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:13.267618 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:13.268780 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:13.268780 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:13.268780 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:13.268780 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:13.268780 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:13.268780 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:13.268780 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:13.268780 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:13.268780 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:13.268780 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:13.268780 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:29:13.268780 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:13.268780 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 407/811] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 15:29:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:13.269646 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ba193-6ac2-4e3b-9c87-43da4f7d5361 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:13.269646 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ba193-6ac2-4e3b-9c87-43da4f7d5361 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:13.269646 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ba193-6ac2-4e3b-9c87-43da4f7d5361 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:13.275975 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 408/812] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:13.276879 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8ba193-6ac2-4e3b-9c87-43da4f7d5361 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:13.347704 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:13.354665 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:13.363053 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e5bbc-4ad5-49b9-83ab-e0a425c21880 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:13.388380 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8ba193-6ac2-4e3b-9c87-43da4f7d5361 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:13.400386 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d773e3-c087-4472-81cf-6800106a4762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:13.400994 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d773e3-c087-4472-81cf-6800106a4762 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:13.401370 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d773e3-c087-4472-81cf-6800106a4762 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:13.401536 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d773e3-c087-4472-81cf-6800106a4762 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:13.402486 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d773e3-c087-4472-81cf-6800106a4762 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:13.403574 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d773e3-c087-4472-81cf-6800106a4762 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:13.427552 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 405/813] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:29:13 2022] POST /identity/v3/endpoints => generated 337 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:29:13.427552 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d773e3-c087-4472-81cf-6800106a4762 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:13.436707 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa131bfe-0694-41a8-9c50-3837341f33aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:13.437347 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa131bfe-0694-41a8-9c50-3837341f33aa None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:13.437622 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa131bfe-0694-41a8-9c50-3837341f33aa None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:13.437945 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa131bfe-0694-41a8-9c50-3837341f33aa None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:13.438617 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa131bfe-0694-41a8-9c50-3837341f33aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:13.438921 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa131bfe-0694-41a8-9c50-3837341f33aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:13.440391 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa131bfe-0694-41a8-9c50-3837341f33aa None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:13.454972 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 409/814] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:13.456907 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c1fab-d15e-4063-80de-69d9ea2999c8 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:13.456907 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c1fab-d15e-4063-80de-69d9ea2999c8 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:13.456907 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c1fab-d15e-4063-80de-69d9ea2999c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:13.481755 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-905c1fab-d15e-4063-80de-69d9ea2999c8 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:13.483308 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-905c1fab-d15e-4063-80de-69d9ea2999c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:13.572390 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 410/815] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:13 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:29:13.573359 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c1fab-d15e-4063-80de-69d9ea2999c8 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:13.670946 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:13.691694 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d773e3-c087-4472-81cf-6800106a4762 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:13.720406 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c1fab-d15e-4063-80de-69d9ea2999c8 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:13.740725 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002ef45-f5b2-4143-82cf-f8d20a4a9919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:13.742045 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002ef45-f5b2-4143-82cf-f8d20a4a9919 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:13.742483 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002ef45-f5b2-4143-82cf-f8d20a4a9919 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:13.742893 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002ef45-f5b2-4143-82cf-f8d20a4a9919 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:13.743688 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b002ef45-f5b2-4143-82cf-f8d20a4a9919 None admin] RBAC: Adding query filter params (name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:13.744141 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b002ef45-f5b2-4143-82cf-f8d20a4a9919 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:13.745666 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b002ef45-f5b2-4143-82cf-f8d20a4a9919 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:13.762021 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 406/816] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 15:29:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:13.763213 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d29912-0e9f-4e6e-ba0d-9d8dbeeb3c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:13.763822 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002ef45-f5b2-4143-82cf-f8d20a4a9919 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:13.764320 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d29912-0e9f-4e6e-ba0d-9d8dbeeb3c51 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:13.764805 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d29912-0e9f-4e6e-ba0d-9d8dbeeb3c51 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:13.765283 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d29912-0e9f-4e6e-ba0d-9d8dbeeb3c51 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:13.776151 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d29912-0e9f-4e6e-ba0d-9d8dbeeb3c51 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:13.778485 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d29912-0e9f-4e6e-ba0d-9d8dbeeb3c51 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:13.780389 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:13.798450 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-14d29912-0e9f-4e6e-ba0d-9d8dbeeb3c51 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:13.798450 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:13.798450 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:13.798450 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:13.798450 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:13.798450 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:13.798450 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:13.798450 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:13.798450 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:13.798450 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:13.798450 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:13.798450 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:13.798450 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:29:13.798450 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:29:13.798450 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:29:13.799386 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:29:13.799386 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:13.799386 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:13.799386 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:13.799386 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:13.799386 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:13.799386 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:13.799386 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:13.799386 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:13.799386 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:13.799386 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:13.799386 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:13.799386 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:13.799386 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:13.799386 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:13.799386 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:13.799386 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:13.800280 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:29:13.800280 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:29:13.800280 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:29:13.800280 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:29:13.800280 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:29:13.800280 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:29:13.800280 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:13.800280 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:13.803152 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 411/817] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:29:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:13.808703 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d29912-0e9f-4e6e-ba0d-9d8dbeeb3c51 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:13.892902 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002ef45-f5b2-4143-82cf-f8d20a4a9919 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:13.924791 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d29912-0e9f-4e6e-ba0d-9d8dbeeb3c51 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:13.935430 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2708320-edc6-4c53-a5bc-47fb6cf640ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:13.936170 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2708320-edc6-4c53-a5bc-47fb6cf640ef None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:13.936546 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2708320-edc6-4c53-a5bc-47fb6cf640ef None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:13.936936 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2708320-edc6-4c53-a5bc-47fb6cf640ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:13.939837 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:13.947428 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2708320-edc6-4c53-a5bc-47fb6cf640ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:13.949003 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2708320-edc6-4c53-a5bc-47fb6cf640ef None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:13.964094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-c2708320-edc6-4c53-a5bc-47fb6cf640ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:13.964094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:13.964094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:13.964094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:13.964094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:13.964094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:13.964094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:13.964094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:13.964094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:13.964094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:13.964094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:13.964094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:13.964094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:13.964094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:13.964094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:13.964094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:13.965234 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:13.965234 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:13.965234 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:13.965234 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:13.965234 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:13.965234 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:13.965234 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:13.965234 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:13.965234 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:13.965234 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:13.965234 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:13.965234 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:13.965234 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:13.965234 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:13.965234 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:13.965234 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:13.965234 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:13.966382 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:13.966382 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:13.966382 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:13.966586 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b566df2-2c8e-4299-aeae-2cd909805822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:13.966697 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b566df2-2c8e-4299-aeae-2cd909805822 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:13.967049 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b566df2-2c8e-4299-aeae-2cd909805822 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:13.967400 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b566df2-2c8e-4299-aeae-2cd909805822 None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:13.968169 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b566df2-2c8e-4299-aeae-2cd909805822 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:13.968559 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b566df2-2c8e-4299-aeae-2cd909805822 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:13.970082 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b566df2-2c8e-4299-aeae-2cd909805822 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:13.971059 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 407/818] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:13.971817 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2708320-edc6-4c53-a5bc-47fb6cf640ef None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:13.983807 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 412/819] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 15:29:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:13.984888 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b566df2-2c8e-4299-aeae-2cd909805822 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:14.044497 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:14.086495 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2708320-edc6-4c53-a5bc-47fb6cf640ef None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:14.090387 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b566df2-2c8e-4299-aeae-2cd909805822 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:14.122860 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97563f12-06b9-446e-9978-eed5d9e66644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:14.123810 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97563f12-06b9-446e-9978-eed5d9e66644 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:14.123972 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e10898-eeff-4df9-aa3b-fb693e9181fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:14.124477 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97563f12-06b9-446e-9978-eed5d9e66644 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:14.124589 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e10898-eeff-4df9-aa3b-fb693e9181fc None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:14.124827 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e10898-eeff-4df9-aa3b-fb693e9181fc None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:14.125092 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e10898-eeff-4df9-aa3b-fb693e9181fc None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:14.125253 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97563f12-06b9-446e-9978-eed5d9e66644 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:14.125726 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e10898-eeff-4df9-aa3b-fb693e9181fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:14.126064 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e10898-eeff-4df9-aa3b-fb693e9181fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:14.126244 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97563f12-06b9-446e-9978-eed5d9e66644 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:14.127422 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e10898-eeff-4df9-aa3b-fb693e9181fc None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:14.127855 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97563f12-06b9-446e-9978-eed5d9e66644 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:14.142574 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-97563f12-06b9-446e-9978-eed5d9e66644 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 28 15:29:14.142574 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:14.142574 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:29:14.142574 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:29:14.142574 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:29:14.142574 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:29:14.142574 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:29:14.142574 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:29:14.142574 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 28 15:29:14.142574 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:14.142574 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:29:14.142574 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:14.142574 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:14.142574 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:14.142574 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:14.142574 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:14.142574 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:14.143818 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:14.143818 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:14.143818 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:14.143818 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:14.143818 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:14.143818 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:14.143818 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:29:14.143818 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:29:14.143818 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:29:14.143818 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:29:14.143818 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:14.143818 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:14.143818 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:14.143818 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:14.143818 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:14.143818 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:14.144856 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:14.144856 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:14.144856 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:14.144856 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:14.144856 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:14.144856 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:14.144856 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:14.144856 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:14.144856 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:14.144856 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:14.144856 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:29:14.144856 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:29:14.144856 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 28 15:29:14.144856 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:14.146154 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 413/820] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:14.146952 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e10898-eeff-4df9-aa3b-fb693e9181fc None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:14.152379 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 408/821] 10.222.0.52 () {56 vars in 1129 bytes} [Wed Sep 28 15:29:13 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:14.153357 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97563f12-06b9-446e-9978-eed5d9e66644 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:14.187022 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:14.256508 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e10898-eeff-4df9-aa3b-fb693e9181fc None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:14.257074 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97563f12-06b9-446e-9978-eed5d9e66644 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:14.292062 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da24b7b0-5e36-4114-8db0-490220fad90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:14.292981 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da24b7b0-5e36-4114-8db0-490220fad90e None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:14.293399 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da24b7b0-5e36-4114-8db0-490220fad90e None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:14.293822 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da24b7b0-5e36-4114-8db0-490220fad90e None admin] PATH_INFO: `/v3/projects/3dc6c5f5fcd34922b0d09cd2aaed6cf1/users/328051fe7607464f8f5dc0dcc01a817a/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:14.295965 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da24b7b0-5e36-4114-8db0-490220fad90e None admin] RBAC: Authorizing `identity:create_grant(project_id=3dc6c5f5fcd34922b0d09cd2aaed6cf1, user_id=328051fe7607464f8f5dc0dcc01a817a, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:14.297414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da24b7b0-5e36-4114-8db0-490220fad90e None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:14.298072 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:14.309154 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108fc795-47e5-4e7a-8021-1f079b951187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:14.309906 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108fc795-47e5-4e7a-8021-1f079b951187 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:14.310249 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108fc795-47e5-4e7a-8021-1f079b951187 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:14.310583 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108fc795-47e5-4e7a-8021-1f079b951187 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:14.322285 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 414/822] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 15:29:14 2022] PUT /identity/v3/projects/3dc6c5f5fcd34922b0d09cd2aaed6cf1/users/328051fe7607464f8f5dc0dcc01a817a/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:29:14.322285 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da24b7b0-5e36-4114-8db0-490220fad90e None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:14.322594 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108fc795-47e5-4e7a-8021-1f079b951187 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:14.322594 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108fc795-47e5-4e7a-8021-1f079b951187 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:14.334715 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-108fc795-47e5-4e7a-8021-1f079b951187 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 15:29:14.334715 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:14.334715 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:14.334715 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:14.334715 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:14.334715 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:14.334715 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:14.334715 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:14.334715 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:14.334715 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:14.334715 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:14.334715 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:14.334715 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:29:14.334715 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:29:14.334715 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:29:14.334715 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:29:14.335969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:14.335969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:14.335969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:29:14.335969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:14.335969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:29:14.335969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:14.335969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:14.335969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:14.335969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:14.335969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:14.335969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:14.335969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:14.335969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:14.335969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:14.335969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:14.335969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:14.335969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:14.335969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:14.337137 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:14.337137 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:14.337137 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:29:14.337137 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:29:14.337137 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:29:14.337137 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:29:14.337137 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:29:14.337137 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:29:14.337137 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 15:29:14.337137 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:14.337983 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 409/823] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Sep 28 15:29:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:14.343028 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108fc795-47e5-4e7a-8021-1f079b951187 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:14.448342 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:14.455958 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da24b7b0-5e36-4114-8db0-490220fad90e None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:14.488453 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108fc795-47e5-4e7a-8021-1f079b951187 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:14.498480 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aa1046-1234-41ec-9ac2-0be56f3be7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:14.499094 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aa1046-1234-41ec-9ac2-0be56f3be7fa None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:14.499372 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aa1046-1234-41ec-9ac2-0be56f3be7fa None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:14.499644 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aa1046-1234-41ec-9ac2-0be56f3be7fa None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:14.500270 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aa1046-1234-41ec-9ac2-0be56f3be7fa None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:14.500571 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aa1046-1234-41ec-9ac2-0be56f3be7fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:14.502351 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aa1046-1234-41ec-9ac2-0be56f3be7fa None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:14.508263 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 415/824] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Sep 28 15:29:14 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:14.514265 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aa1046-1234-41ec-9ac2-0be56f3be7fa None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:14.531429 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9be6af-9143-4f2b-a517-2e4e87ecc0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:14.532121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9be6af-9143-4f2b-a517-2e4e87ecc0a0 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:14.532311 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9be6af-9143-4f2b-a517-2e4e87ecc0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:14.532580 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9be6af-9143-4f2b-a517-2e4e87ecc0a0 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:14.533285 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9be6af-9143-4f2b-a517-2e4e87ecc0a0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:14.533583 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9be6af-9143-4f2b-a517-2e4e87ecc0a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:14.535039 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9be6af-9143-4f2b-a517-2e4e87ecc0a0 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:14.558410 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 410/825] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Sep 28 15:29:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:14.563655 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9be6af-9143-4f2b-a517-2e4e87ecc0a0 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:14.615085 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:14.649512 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aa1046-1234-41ec-9ac2-0be56f3be7fa None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:14.677159 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9be6af-9143-4f2b-a517-2e4e87ecc0a0 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:14.689431 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07587218-a165-4589-abc5-071b322d3171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:14.690745 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07587218-a165-4589-abc5-071b322d3171 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:14.691250 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07587218-a165-4589-abc5-071b322d3171 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:14.691733 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07587218-a165-4589-abc5-071b322d3171 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:14.692566 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07587218-a165-4589-abc5-071b322d3171 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:14.693077 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07587218-a165-4589-abc5-071b322d3171 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:14.695324 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07587218-a165-4589-abc5-071b322d3171 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:14.701922 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 416/826] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Sep 28 15:29:14 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:14.711699 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07587218-a165-4589-abc5-071b322d3171 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:14.721044 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc6b9f6-05a7-4b87-8b48-08be7be68ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:14.721682 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6b9f6-05a7-4b87-8b48-08be7be68ea3 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:14.722004 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6b9f6-05a7-4b87-8b48-08be7be68ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:14.722331 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc6b9f6-05a7-4b87-8b48-08be7be68ea3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:14.733313 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc6b9f6-05a7-4b87-8b48-08be7be68ea3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:14.734945 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc6b9f6-05a7-4b87-8b48-08be7be68ea3 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:14.741736 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:14.750100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-bcc6b9f6-05a7-4b87-8b48-08be7be68ea3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:14.750100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:14.750100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:14.750100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:14.750100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:14.750100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:14.750100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:14.750100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:14.750100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:14.750100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:14.750100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:14.750100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:14.750100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:14.750100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:14.750100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:14.750100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:14.751333 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:14.751333 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:14.751333 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:14.751333 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:14.751333 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:14.751333 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:14.751333 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:14.751333 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:14.751333 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:14.751333 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:14.751333 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:14.751333 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:14.751333 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:14.751333 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:14.751333 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:14.751333 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:14.751333 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:14.752219 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:14.752219 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:14.752219 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:14.752219 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 411/827] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:29:14.758777 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc6b9f6-05a7-4b87-8b48-08be7be68ea3 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:14.826268 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07587218-a165-4589-abc5-071b322d3171 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:14.860728 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1413e9ac-c42b-4c21-af0d-60ce3b3da2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:14.861456 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1413e9ac-c42b-4c21-af0d-60ce3b3da2c2 None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:14.861794 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1413e9ac-c42b-4c21-af0d-60ce3b3da2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:14.862208 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1413e9ac-c42b-4c21-af0d-60ce3b3da2c2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:14.862946 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1413e9ac-c42b-4c21-af0d-60ce3b3da2c2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:14.864464 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1413e9ac-c42b-4c21-af0d-60ce3b3da2c2 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:14.870186 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:14.874482 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc6b9f6-05a7-4b87-8b48-08be7be68ea3 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:14.889155 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 417/828] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:29:14 2022] POST /identity/v3/endpoints => generated 346 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:29:14.925211 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa50a37-f8a0-4a2a-8372-856d03774cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:14.925963 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa50a37-f8a0-4a2a-8372-856d03774cbf None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:14.926243 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa50a37-f8a0-4a2a-8372-856d03774cbf None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:14.926573 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa50a37-f8a0-4a2a-8372-856d03774cbf None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:14.927215 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa50a37-f8a0-4a2a-8372-856d03774cbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:14.927569 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa50a37-f8a0-4a2a-8372-856d03774cbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:14.929838 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa50a37-f8a0-4a2a-8372-856d03774cbf None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:14.944263 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 412/829] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:14.949942 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1413e9ac-c42b-4c21-af0d-60ce3b3da2c2 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:15.042023 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:15.065787 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1413e9ac-c42b-4c21-af0d-60ce3b3da2c2 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:15.102152 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb03fdb-4395-42b0-b944-2c7ce481c195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:15.102781 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb03fdb-4395-42b0-b944-2c7ce481c195 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:15.103110 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb03fdb-4395-42b0-b944-2c7ce481c195 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:15.103378 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb03fdb-4395-42b0-b944-2c7ce481c195 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:15.110338 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb03fdb-4395-42b0-b944-2c7ce481c195 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:15.111775 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb03fdb-4395-42b0-b944-2c7ce481c195 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:15.123418 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-5eb03fdb-4395-42b0-b944-2c7ce481c195 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:15.123418 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:15.123418 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:15.123418 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:15.123418 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:15.123418 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:15.123418 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:15.123418 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:15.123418 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:15.123418 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:15.123418 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:15.123418 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:15.123418 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:29:15.123418 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:29:15.123418 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:29:15.124312 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:29:15.124312 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:15.124312 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:15.124312 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:15.124312 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:15.124312 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:15.124312 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:15.124312 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:15.124312 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:15.124312 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:15.124312 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:15.124312 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:15.124312 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:15.124312 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:15.124312 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:15.124312 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:15.124312 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:15.125216 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:29:15.125216 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:29:15.125216 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:29:15.125216 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:29:15.125216 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:29:15.125216 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:29:15.125216 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:15.125216 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:15.125216 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 418/830] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:29:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:15.130675 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa50a37-f8a0-4a2a-8372-856d03774cbf None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:15.240667 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa50a37-f8a0-4a2a-8372-856d03774cbf None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:15.277717 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c57734-f2ab-493b-b1af-5645071e4125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:15.278776 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c57734-f2ab-493b-b1af-5645071e4125 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:15.279260 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c57734-f2ab-493b-b1af-5645071e4125 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:15.279750 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c57734-f2ab-493b-b1af-5645071e4125 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:15.280692 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c57734-f2ab-493b-b1af-5645071e4125 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:15.281210 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c57734-f2ab-493b-b1af-5645071e4125 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:15.282913 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c57734-f2ab-493b-b1af-5645071e4125 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:15.297720 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 413/831] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 15:29:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:15.302770 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb03fdb-4395-42b0-b944-2c7ce481c195 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:15.408816 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb03fdb-4395-42b0-b944-2c7ce481c195 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:15.444017 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49999e7-4fe2-480d-94c5-514f1cc16986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:15.444791 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49999e7-4fe2-480d-94c5-514f1cc16986 None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:15.445071 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49999e7-4fe2-480d-94c5-514f1cc16986 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:15.445347 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49999e7-4fe2-480d-94c5-514f1cc16986 None admin] PATH_INFO: `/v3/projects/3dc6c5f5fcd34922b0d09cd2aaed6cf1/users/c81e5fb007614cf2992f5e02e85d6307/roles/342b64fe0cdb4925b23c6cd461a152ac` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:15.453815 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:15.460472 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49999e7-4fe2-480d-94c5-514f1cc16986 None admin] RBAC: Authorizing `identity:create_grant(project_id=3dc6c5f5fcd34922b0d09cd2aaed6cf1, user_id=c81e5fb007614cf2992f5e02e85d6307, role_id=342b64fe0cdb4925b23c6cd461a152ac)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:15.461929 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49999e7-4fe2-480d-94c5-514f1cc16986 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:15.478393 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 419/832] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 15:29:15 2022] PUT /identity/v3/projects/3dc6c5f5fcd34922b0d09cd2aaed6cf1/users/c81e5fb007614cf2992f5e02e85d6307/roles/342b64fe0cdb4925b23c6cd461a152ac => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:29:15.623704 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3626005-f743-48a8-a02f-812d9b1d36d7 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:15.624062 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3626005-f743-48a8-a02f-812d9b1d36d7 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:15.624247 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3626005-f743-48a8-a02f-812d9b1d36d7 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:15.625919 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 414/833] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:15.631888 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90e728-42cb-4003-bc95-b29367e5cf9a None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:15.632183 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90e728-42cb-4003-bc95-b29367e5cf9a None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:15.632438 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90e728-42cb-4003-bc95-b29367e5cf9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:15.651589 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-4b90e728-42cb-4003-bc95-b29367e5cf9a None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:15.652766 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4b90e728-42cb-4003-bc95-b29367e5cf9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:15.724610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 420/834] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:15 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:29:15.994286 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3626005-f743-48a8-a02f-812d9b1d36d7 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:16.011269 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:16.099958 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3626005-f743-48a8-a02f-812d9b1d36d7 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:16.134995 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0299ab1-a4d3-4965-a42e-534ecd086fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:16.135618 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0299ab1-a4d3-4965-a42e-534ecd086fb0 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:16.135892 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0299ab1-a4d3-4965-a42e-534ecd086fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:16.136157 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0299ab1-a4d3-4965-a42e-534ecd086fb0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:16.136724 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0299ab1-a4d3-4965-a42e-534ecd086fb0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:16.138098 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0299ab1-a4d3-4965-a42e-534ecd086fb0 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:16.146871 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-a0299ab1-a4d3-4965-a42e-534ecd086fb0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 28 15:29:16.146871 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:16.146871 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:29:16.146871 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:29:16.146871 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:29:16.146871 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:29:16.146871 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:29:16.146871 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:29:16.146871 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 28 15:29:16.146871 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:16.146871 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:29:16.146871 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:16.146871 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:16.146871 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:16.146871 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:16.146871 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:16.148375 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:16.148375 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:16.148375 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:16.148375 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:16.148375 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:16.148375 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:16.148375 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:16.148375 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:29:16.148375 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:29:16.148375 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:29:16.148375 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:29:16.148375 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:16.148375 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:16.148375 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:16.148375 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:16.148375 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:16.148375 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:16.149592 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:16.149592 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:16.149592 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:16.149592 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:16.149592 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:16.149592 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:16.149592 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:16.149592 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:16.149592 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:16.149592 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:16.149592 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:29:16.149592 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:29:16.149592 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 28 15:29:16.149592 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:16.149592 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 415/835] 10.222.0.52 () {56 vars in 1132 bytes} [Wed Sep 28 15:29:15 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:16.154562 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90e728-42cb-4003-bc95-b29367e5cf9a None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:16.268404 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90e728-42cb-4003-bc95-b29367e5cf9a None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:16.287725 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:16.307869 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b77ea1-7ab3-42b6-a2ed-70fbea76ab9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:16.308463 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b77ea1-7ab3-42b6-a2ed-70fbea76ab9a None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:16.308741 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b77ea1-7ab3-42b6-a2ed-70fbea76ab9a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:16.309896 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b77ea1-7ab3-42b6-a2ed-70fbea76ab9a None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:16.309896 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b77ea1-7ab3-42b6-a2ed-70fbea76ab9a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:16.309896 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b77ea1-7ab3-42b6-a2ed-70fbea76ab9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:16.311263 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b77ea1-7ab3-42b6-a2ed-70fbea76ab9a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:16.316332 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 421/836] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Sep 28 15:29:16 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:16.322924 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0299ab1-a4d3-4965-a42e-534ecd086fb0 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:16.440188 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:16.477807 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0299ab1-a4d3-4965-a42e-534ecd086fb0 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:16.540900 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daede509-b6a6-4170-beec-439e4b3b484d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:16.541792 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daede509-b6a6-4170-beec-439e4b3b484d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:16.542305 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daede509-b6a6-4170-beec-439e4b3b484d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:16.542775 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daede509-b6a6-4170-beec-439e4b3b484d None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:16.543860 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daede509-b6a6-4170-beec-439e4b3b484d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:16.544378 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daede509-b6a6-4170-beec-439e4b3b484d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:16.546923 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daede509-b6a6-4170-beec-439e4b3b484d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:16.555897 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 416/837] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Sep 28 15:29:16 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:16.644461 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6455c1b5-ad9a-47f4-8ac7-daff4d278148 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:16.644868 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6455c1b5-ad9a-47f4-8ac7-daff4d278148 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:16.645280 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6455c1b5-ad9a-47f4-8ac7-daff4d278148 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:16.648577 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 422/838] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:16.655280 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27016974-9ba9-4bcc-85a4-0a3478d9bf3b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:16.656238 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27016974-9ba9-4bcc-85a4-0a3478d9bf3b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:16.656864 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27016974-9ba9-4bcc-85a4-0a3478d9bf3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:16.677171 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-27016974-9ba9-4bcc-85a4-0a3478d9bf3b None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:16.678413 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-27016974-9ba9-4bcc-85a4-0a3478d9bf3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:16.754716 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 417/839] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:16 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:29:17.033040 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6455c1b5-ad9a-47f4-8ac7-daff4d278148 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:17.139311 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6455c1b5-ad9a-47f4-8ac7-daff4d278148 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:17.173895 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a33022-ffbd-4c61-96e8-70bfb2c2fa6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:17.174519 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a33022-ffbd-4c61-96e8-70bfb2c2fa6f None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:17.174788 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a33022-ffbd-4c61-96e8-70bfb2c2fa6f None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:17.175050 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a33022-ffbd-4c61-96e8-70bfb2c2fa6f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:17.185370 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a33022-ffbd-4c61-96e8-70bfb2c2fa6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:17.190310 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-b5a33022-ffbd-4c61-96e8-70bfb2c2fa6f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:29:17.190310 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:17.190310 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:17.190310 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:17.190310 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:17.190310 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:17.190310 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:17.190310 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:17.190310 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:17.190310 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:17.190310 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:17.190310 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:17.190310 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:17.190310 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:17.190310 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:17.190310 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:17.191216 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:17.191216 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:17.191216 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:17.191216 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:17.191216 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:17.191216 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:17.191216 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:17.191216 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:17.191216 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:17.191216 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:17.191216 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:17.191216 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:17.191216 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:17.191216 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:17.191216 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:17.191216 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:17.192314 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:17.192314 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:17.192314 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:17.192314 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:17.192314 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:17.192314 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:17.192314 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:17.192314 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:17.192314 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:17.192314 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:17.192314 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:29:17.192314 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:17.192314 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 423/840] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 15:29:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:17.200014 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27016974-9ba9-4bcc-85a4-0a3478d9bf3b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:17.279908 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:17.314255 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27016974-9ba9-4bcc-85a4-0a3478d9bf3b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:17.331828 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58bf329-f058-48b7-8715-eacfdaaf34da None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:17.332135 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58bf329-f058-48b7-8715-eacfdaaf34da None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:17.332392 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58bf329-f058-48b7-8715-eacfdaaf34da None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:17.334619 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 424/841] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:17.339940 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc4f21-e214-422c-920d-ff18e10ff86f None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:17.340218 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc4f21-e214-422c-920d-ff18e10ff86f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:17.340484 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc4f21-e214-422c-920d-ff18e10ff86f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:17.351055 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc92b088-ca2f-4017-a6d9-d065adc8bd6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:17.351667 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92b088-ca2f-4017-a6d9-d065adc8bd6b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:17.351950 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92b088-ca2f-4017-a6d9-d065adc8bd6b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:17.352225 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92b088-ca2f-4017-a6d9-d065adc8bd6b None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:17.352900 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc92b088-ca2f-4017-a6d9-d065adc8bd6b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:17.353202 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc92b088-ca2f-4017-a6d9-d065adc8bd6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:17.354621 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc92b088-ca2f-4017-a6d9-d065adc8bd6b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:17.360732 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-e3dc4f21-e214-422c-920d-ff18e10ff86f None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:17.362147 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e3dc4f21-e214-422c-920d-ff18e10ff86f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:17.363548 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 418/842] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:29:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:17.370875 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc92b088-ca2f-4017-a6d9-d065adc8bd6b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:17.401572 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:17.442865 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 425/843] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:17 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 15:29:17.482588 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc92b088-ca2f-4017-a6d9-d065adc8bd6b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:17.520961 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e5a62b-7ba7-4cc5-8ccb-ab79f9dc0a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:17.521582 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5a62b-7ba7-4cc5-8ccb-ab79f9dc0a1b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:17.521878 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5a62b-7ba7-4cc5-8ccb-ab79f9dc0a1b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:17.522153 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5a62b-7ba7-4cc5-8ccb-ab79f9dc0a1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:17.524073 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:17.533766 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e5a62b-7ba7-4cc5-8ccb-ab79f9dc0a1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:17.535371 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e5a62b-7ba7-4cc5-8ccb-ab79f9dc0a1b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:17.551381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-d9e5a62b-7ba7-4cc5-8ccb-ab79f9dc0a1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:17.551381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:17.551381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:17.551381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:17.551381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:17.551381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:17.551381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:17.551381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:17.551381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:17.551381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:17.551381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:17.551381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:17.551381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:17.551381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:17.551381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:17.551381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:17.552373 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:17.552373 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:17.552373 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:17.552373 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:17.552373 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:17.552373 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:17.552373 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:17.552373 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:17.552373 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:17.552373 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:17.552373 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:17.552373 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:17.552373 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:17.552373 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:17.552373 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:17.552373 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:17.552373 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:17.553422 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:17.553422 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:17.553422 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:17.553422 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 419/844] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:17.560464 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc4f21-e214-422c-920d-ff18e10ff86f None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:17.672023 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc4f21-e214-422c-920d-ff18e10ff86f None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:17.706019 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e580d24-7076-4866-b987-b2d55afceac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:17.706618 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e580d24-7076-4866-b987-b2d55afceac3 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:17.706894 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e580d24-7076-4866-b987-b2d55afceac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:17.707162 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e580d24-7076-4866-b987-b2d55afceac3 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:17.707794 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e580d24-7076-4866-b987-b2d55afceac3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:17.708093 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e580d24-7076-4866-b987-b2d55afceac3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:17.709459 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e580d24-7076-4866-b987-b2d55afceac3 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:17.719408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e5a62b-7ba7-4cc5-8ccb-ab79f9dc0a1b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:17.721781 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 426/845] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:17.730060 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e580d24-7076-4866-b987-b2d55afceac3 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:17.766259 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:17.825491 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e5a62b-7ba7-4cc5-8ccb-ab79f9dc0a1b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:17.839566 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e580d24-7076-4866-b987-b2d55afceac3 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:17.859750 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9d2172-3a44-4e1c-b551-db620ed05a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:17.860375 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d2172-3a44-4e1c-b551-db620ed05a8d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:17.860652 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d2172-3a44-4e1c-b551-db620ed05a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:17.860912 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d2172-3a44-4e1c-b551-db620ed05a8d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:17.861481 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9d2172-3a44-4e1c-b551-db620ed05a8d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:17.862884 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9d2172-3a44-4e1c-b551-db620ed05a8d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:17.872125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-0c9d2172-3a44-4e1c-b551-db620ed05a8d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 28 15:29:17.872125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:17.872125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:29:17.872125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:29:17.872125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:29:17.872125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:29:17.872125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:29:17.872125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:29:17.872125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 28 15:29:17.872125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:17.872125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:29:17.872125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:17.872125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:17.872125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:17.872125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:17.872125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:17.872125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:17.873117 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:17.873117 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:17.873117 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:17.873117 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:17.873117 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:17.873117 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:17.873117 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:29:17.873117 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:29:17.873117 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:29:17.873117 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:29:17.873117 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:17.873117 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:17.873117 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:17.873117 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:17.873117 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:17.873117 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:17.873997 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:17.873997 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:17.873997 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:17.873997 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:17.873997 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:17.873997 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:17.873997 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:17.873997 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:17.873997 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:17.873997 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:17.873997 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:29:17.873997 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:29:17.873997 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 28 15:29:17.873997 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:17.878093 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 420/846] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 15:29:17 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:29:17.878944 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9d2172-3a44-4e1c-b551-db620ed05a8d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:17.881012 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59de79ee-33de-4a7e-8f6e-de7d576b2bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:17.881665 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de79ee-33de-4a7e-8f6e-de7d576b2bf5 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:17.881976 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de79ee-33de-4a7e-8f6e-de7d576b2bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:17.882245 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de79ee-33de-4a7e-8f6e-de7d576b2bf5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:17.889961 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59de79ee-33de-4a7e-8f6e-de7d576b2bf5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:17.891399 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59de79ee-33de-4a7e-8f6e-de7d576b2bf5 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:17.903892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-59de79ee-33de-4a7e-8f6e-de7d576b2bf5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 28 15:29:17.903892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:17.903892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:17.903892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:17.903892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:17.903892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:17.903892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:17.903892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:17.903892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:17.903892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:17.903892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:17.903892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:17.903892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:29:17.903892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:29:17.903892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:29:17.903892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:29:17.904858 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:17.904858 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:17.904858 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:29:17.904858 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:17.904858 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:29:17.904858 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:17.904858 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:17.904858 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:17.904858 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:17.904858 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:17.904858 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:17.904858 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:17.904858 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:17.904858 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:17.904858 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:17.904858 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:17.904858 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:17.904858 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:17.905800 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:17.905800 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:17.905800 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:29:17.905800 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:29:17.905800 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:29:17.905800 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:29:17.905800 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:29:17.905800 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:29:17.905800 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 28 15:29:17.905800 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:17.910005 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 427/847] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 15:29:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:17.911294 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59de79ee-33de-4a7e-8f6e-de7d576b2bf5 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:17.935832 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:17.990585 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9d2172-3a44-4e1c-b551-db620ed05a8d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:18.010780 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:18.022056 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59de79ee-33de-4a7e-8f6e-de7d576b2bf5 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:18.028517 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecf3a78-5d17-4803-a0ff-5d766d839993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:18.029107 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf3a78-5d17-4803-a0ff-5d766d839993 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:18.029386 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf3a78-5d17-4803-a0ff-5d766d839993 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:18.029658 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf3a78-5d17-4803-a0ff-5d766d839993 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:18.030306 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecf3a78-5d17-4803-a0ff-5d766d839993 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:18.030605 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecf3a78-5d17-4803-a0ff-5d766d839993 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:18.032111 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecf3a78-5d17-4803-a0ff-5d766d839993 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:18.041705 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 421/848] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 15:29:17 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:18.041705 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1282e64a-09e4-41b6-9af9-9b973c23978a None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:18.041705 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1282e64a-09e4-41b6-9af9-9b973c23978a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:18.041705 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1282e64a-09e4-41b6-9af9-9b973c23978a None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:18.041705 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 422/849] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:18.045252 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1282e64a-09e4-41b6-9af9-9b973c23978a None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:18.068367 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cffe6f-0e20-4860-be34-02f828f6c877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:18.068963 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cffe6f-0e20-4860-be34-02f828f6c877 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:18.069277 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cffe6f-0e20-4860-be34-02f828f6c877 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:18.069600 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cffe6f-0e20-4860-be34-02f828f6c877 None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:18.070285 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cffe6f-0e20-4860-be34-02f828f6c877 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:18.070581 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cffe6f-0e20-4860-be34-02f828f6c877 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:18.072106 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cffe6f-0e20-4860-be34-02f828f6c877 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:18.095960 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 428/850] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Sep 28 15:29:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:18.095960 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db2df8-4239-42a2-a5b6-5d4eba9098d7 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:18.095960 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db2df8-4239-42a2-a5b6-5d4eba9098d7 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:18.095960 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db2df8-4239-42a2-a5b6-5d4eba9098d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:18.112018 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-20db2df8-4239-42a2-a5b6-5d4eba9098d7 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:18.113122 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-20db2df8-4239-42a2-a5b6-5d4eba9098d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:18.154992 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1282e64a-09e4-41b6-9af9-9b973c23978a None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:18.169564 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:18.188536 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 429/851] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:18 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:29:18.190003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20db2df8-4239-42a2-a5b6-5d4eba9098d7 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:18.197286 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c0dc1-4722-4162-a79b-8a5c5bd15000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:18.198195 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c0dc1-4722-4162-a79b-8a5c5bd15000 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:18.198684 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c0dc1-4722-4162-a79b-8a5c5bd15000 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:18.199224 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c0dc1-4722-4162-a79b-8a5c5bd15000 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:18.200108 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46c0dc1-4722-4162-a79b-8a5c5bd15000 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:18.201120 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46c0dc1-4722-4162-a79b-8a5c5bd15000 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:18.202819 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46c0dc1-4722-4162-a79b-8a5c5bd15000 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:18.208463 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 423/852] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 15:29:18 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:18.287182 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:18.312213 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20db2df8-4239-42a2-a5b6-5d4eba9098d7 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:18.349994 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025d97cc-959f-4f83-a7d5-a051d19d0e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:18.350627 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025d97cc-959f-4f83-a7d5-a051d19d0e3f None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:18.350901 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025d97cc-959f-4f83-a7d5-a051d19d0e3f None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:18.351170 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025d97cc-959f-4f83-a7d5-a051d19d0e3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:18.430108 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025d97cc-959f-4f83-a7d5-a051d19d0e3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:18.431592 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025d97cc-959f-4f83-a7d5-a051d19d0e3f None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:18.474267 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c0dc1-4722-4162-a79b-8a5c5bd15000 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:18.542217 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:18.546764 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 430/853] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:29:18 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 15:29:18.548053 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025d97cc-959f-4f83-a7d5-a051d19d0e3f None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:18.587889 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c0dc1-4722-4162-a79b-8a5c5bd15000 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:18.627185 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5ccb2-69f4-441e-b903-c7f438c59bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:18.627868 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5ccb2-69f4-441e-b903-c7f438c59bd8 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:18.627868 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5ccb2-69f4-441e-b903-c7f438c59bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:18.628051 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5ccb2-69f4-441e-b903-c7f438c59bd8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:18.638981 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e5ccb2-69f4-441e-b903-c7f438c59bd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:18.643832 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-46e5ccb2-69f4-441e-b903-c7f438c59bd8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:29:18.643832 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:18.643832 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:18.643832 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:18.643832 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:18.643832 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:18.643832 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:18.643832 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:18.643832 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:18.643832 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:18.643832 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:18.643832 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:18.643832 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:18.643832 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:18.643832 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:18.643832 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:18.644746 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:18.644746 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:18.644746 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:18.644746 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:18.644746 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:18.644746 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:18.644746 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:18.644746 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:18.644746 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:18.644746 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:18.644746 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:18.644746 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:18.644746 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:18.644746 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:18.644746 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:18.644746 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:18.645568 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:18.645568 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:18.645568 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:18.645568 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:18.645568 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:18.645568 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:18.645568 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:18.645568 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:18.645568 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:18.645568 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:18.645568 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:29:18.645568 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:18.645568 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 424/854] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 15:29:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:29:18.651155 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5ccb2-69f4-441e-b903-c7f438c59bd8 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:18.661109 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025d97cc-959f-4f83-a7d5-a051d19d0e3f None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:18.703799 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:18.706086 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c6e51d-b668-4f6e-9edb-0da240a42240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:18.706690 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6e51d-b668-4f6e-9edb-0da240a42240 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:18.706963 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6e51d-b668-4f6e-9edb-0da240a42240 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:18.707230 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6e51d-b668-4f6e-9edb-0da240a42240 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:18.721290 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c6e51d-b668-4f6e-9edb-0da240a42240 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:18.722771 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c6e51d-b668-4f6e-9edb-0da240a42240 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:18.739423 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-66c6e51d-b668-4f6e-9edb-0da240a42240 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:18.739423 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:18.739423 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:18.739423 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:18.739423 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:18.739423 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:18.739423 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:18.739423 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:18.739423 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:18.739423 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:18.739423 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:18.739423 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:18.739423 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:18.739423 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:18.739423 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:18.739423 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:18.740408 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:18.740408 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:18.740408 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:18.740408 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:18.740408 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:18.740408 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:18.740408 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:18.740408 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:18.740408 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:18.740408 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:18.740408 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:18.740408 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:18.740408 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:18.740408 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:18.740408 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:18.740408 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:18.740408 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:18.741919 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:18.741919 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:18.741919 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:18.748349 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 431/855] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:29:18.749870 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c6e51d-b668-4f6e-9edb-0da240a42240 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:18.782592 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5ccb2-69f4-441e-b903-c7f438c59bd8 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:18.786365 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:18.823766 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f9b66e-7305-4bea-9a04-98b9188bd729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:18.824372 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f9b66e-7305-4bea-9a04-98b9188bd729 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:18.824651 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f9b66e-7305-4bea-9a04-98b9188bd729 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:18.824971 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f9b66e-7305-4bea-9a04-98b9188bd729 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:18.825564 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f9b66e-7305-4bea-9a04-98b9188bd729 None admin] RBAC: Adding query filter params (name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:18.825874 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f9b66e-7305-4bea-9a04-98b9188bd729 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:18.827257 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f9b66e-7305-4bea-9a04-98b9188bd729 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:18.840403 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 425/856] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 15:29:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:18.842385 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f9b66e-7305-4bea-9a04-98b9188bd729 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:18.870749 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c6e51d-b668-4f6e-9edb-0da240a42240 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:18.911332 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2644f278-33c6-4f7e-bff7-39f6db3f6377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:18.911899 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644f278-33c6-4f7e-bff7-39f6db3f6377 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:18.912175 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644f278-33c6-4f7e-bff7-39f6db3f6377 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:18.912721 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644f278-33c6-4f7e-bff7-39f6db3f6377 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:18.913685 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2644f278-33c6-4f7e-bff7-39f6db3f6377 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:18.914015 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2644f278-33c6-4f7e-bff7-39f6db3f6377 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:18.915364 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2644f278-33c6-4f7e-bff7-39f6db3f6377 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:18.927025 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 432/857] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:18.928118 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d8c38-dfa3-43c3-8672-6580f6bbb620 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:18.928398 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d8c38-dfa3-43c3-8672-6580f6bbb620 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:18.928697 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d8c38-dfa3-43c3-8672-6580f6bbb620 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:18.930050 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 433/858] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:18.934655 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d8c38-dfa3-43c3-8672-6580f6bbb620 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:18.964713 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:18.968195 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f9b66e-7305-4bea-9a04-98b9188bd729 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:19.004179 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0120f99e-41c2-4807-b431-f44a8818c816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:19.004811 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0120f99e-41c2-4807-b431-f44a8818c816 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:19.005077 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0120f99e-41c2-4807-b431-f44a8818c816 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:19.005751 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0120f99e-41c2-4807-b431-f44a8818c816 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:19.019194 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0120f99e-41c2-4807-b431-f44a8818c816 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:19.020633 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0120f99e-41c2-4807-b431-f44a8818c816 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:19.033233 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:19.035758 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-0120f99e-41c2-4807-b431-f44a8818c816 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:19.035758 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:19.035758 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:19.035758 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:19.035758 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:19.035758 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:19.035758 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:19.035758 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:19.035758 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:19.035758 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:19.035758 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:19.035758 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:19.035758 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:19.035758 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:19.035758 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:19.035758 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:19.036698 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:19.036698 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:19.036698 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:19.036698 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:19.036698 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:19.036698 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:19.036698 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:19.036698 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:19.036698 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:19.036698 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:19.036698 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:19.036698 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:19.036698 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:19.036698 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:19.036698 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:19.036698 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:19.036698 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:19.038852 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:19.038852 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:19.038852 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:19.038852 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 426/859] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:29:19.038852 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c3e4f-eff4-4601-baf2-c455be427542 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:19.039138 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c3e4f-eff4-4601-baf2-c455be427542 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:19.039213 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c3e4f-eff4-4601-baf2-c455be427542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:19.048557 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d8c38-dfa3-43c3-8672-6580f6bbb620 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:19.057014 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-568c3e4f-eff4-4601-baf2-c455be427542 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:19.058229 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-568c3e4f-eff4-4601-baf2-c455be427542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:19.084561 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2f50f9-c6df-4093-8431-47993e429604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:19.086089 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f50f9-c6df-4093-8431-47993e429604 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:19.086623 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f50f9-c6df-4093-8431-47993e429604 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:19.086901 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f50f9-c6df-4093-8431-47993e429604 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:19.095288 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2f50f9-c6df-4093-8431-47993e429604 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:19.096710 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2f50f9-c6df-4093-8431-47993e429604 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:19.110991 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-0b2f50f9-c6df-4093-8431-47993e429604 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:19.110991 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:19.110991 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:19.110991 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:19.110991 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:19.110991 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:19.110991 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:19.110991 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:19.110991 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:19.110991 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:19.110991 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:19.110991 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:19.110991 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:29:19.110991 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:29:19.110991 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:29:19.111913 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:29:19.111913 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:19.111913 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:19.111913 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:19.111913 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:19.111913 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:19.111913 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:19.111913 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:19.111913 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:19.111913 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:19.111913 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:19.111913 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:19.111913 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:19.111913 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:19.111913 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:19.111913 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:19.111913 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:19.118144 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:29:19.118144 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:29:19.118144 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:29:19.118144 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:29:19.118144 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:29:19.118144 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:29:19.118144 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:19.118144 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:19.118144 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 434/860] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:29:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:19.118144 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2f50f9-c6df-4093-8431-47993e429604 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:19.124938 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 427/861] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:19 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:29:19.125720 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568c3e4f-eff4-4601-baf2-c455be427542 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:19.235136 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:19.235742 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2f50f9-c6df-4093-8431-47993e429604 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:19.260953 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568c3e4f-eff4-4601-baf2-c455be427542 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:19.277539 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5240f9f-48b9-41a8-ba5d-582061834bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:19.278490 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5240f9f-48b9-41a8-ba5d-582061834bcd None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:19.279075 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5240f9f-48b9-41a8-ba5d-582061834bcd None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:19.279513 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5240f9f-48b9-41a8-ba5d-582061834bcd None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:19.280313 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5240f9f-48b9-41a8-ba5d-582061834bcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:19.280621 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5240f9f-48b9-41a8-ba5d-582061834bcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:19.286016 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5240f9f-48b9-41a8-ba5d-582061834bcd None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:19.303794 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435eddc8-20cf-443e-b1e1-d26ce684532d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:19.304279 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 435/862] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:19.306017 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435eddc8-20cf-443e-b1e1-d26ce684532d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:19.306541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435eddc8-20cf-443e-b1e1-d26ce684532d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:19.307025 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435eddc8-20cf-443e-b1e1-d26ce684532d None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:19.307957 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435eddc8-20cf-443e-b1e1-d26ce684532d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:19.308591 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435eddc8-20cf-443e-b1e1-d26ce684532d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:19.309733 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5240f9f-48b9-41a8-ba5d-582061834bcd None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:19.310624 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435eddc8-20cf-443e-b1e1-d26ce684532d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:19.326653 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:19.334064 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 428/863] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 15:29:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:19.336952 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435eddc8-20cf-443e-b1e1-d26ce684532d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:19.434418 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5240f9f-48b9-41a8-ba5d-582061834bcd None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:19.454713 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435eddc8-20cf-443e-b1e1-d26ce684532d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:19.468609 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0685c1-7098-4970-9fac-695365a0d857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:19.469230 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0685c1-7098-4970-9fac-695365a0d857 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:19.469499 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0685c1-7098-4970-9fac-695365a0d857 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:19.469760 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0685c1-7098-4970-9fac-695365a0d857 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:19.477676 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0685c1-7098-4970-9fac-695365a0d857 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:19.479131 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0685c1-7098-4970-9fac-695365a0d857 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:19.490237 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9728d-d9bc-45f7-a924-05505a769d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:19.490789 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9728d-d9bc-45f7-a924-05505a769d06 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:19.490889 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-5f0685c1-7098-4970-9fac-695365a0d857 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 15:29:19.490889 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:19.490889 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:19.490889 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:19.490889 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:19.490889 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:19.490889 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:19.490889 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:19.490889 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:19.490889 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:19.490889 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:19.490889 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:19.490889 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:29:19.490889 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:29:19.490889 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:29:19.490889 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:29:19.492335 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:19.492335 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:19.492335 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:29:19.492335 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:19.492335 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:29:19.492335 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:19.492335 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:19.492335 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:19.492335 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:19.492335 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:19.492335 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:19.492335 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:19.492335 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:19.492335 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:19.492335 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:19.492335 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:19.492335 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:19.492335 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:19.493266 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:19.493266 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:19.493266 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:29:19.493266 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:29:19.493266 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:29:19.493266 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:29:19.493266 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:29:19.493266 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:29:19.493266 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 15:29:19.493266 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:19.493793 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9728d-d9bc-45f7-a924-05505a769d06 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:19.493793 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f9728d-d9bc-45f7-a924-05505a769d06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:19.493793 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f9728d-d9bc-45f7-a924-05505a769d06 None admin] RBAC: Adding query filter params (role.id=62d96ac9513c452894fc67b442cfc59f, scope.project.id=3dc6c5f5fcd34922b0d09cd2aaed6cf1, user.id=328051fe7607464f8f5dc0dcc01a817a) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:19.493793 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f9728d-d9bc-45f7-a924-05505a769d06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:19.494615 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f9728d-d9bc-45f7-a924-05505a769d06 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:19.497195 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 436/864] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Sep 28 15:29:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:19.498009 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0685c1-7098-4970-9fac-695365a0d857 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:19.501281 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 429/865] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 15:29:19 2022] GET /identity/v3/role_assignments?user.id=328051fe7607464f8f5dc0dcc01a817a&scope.project.id=3dc6c5f5fcd34922b0d09cd2aaed6cf1&role.id=62d96ac9513c452894fc67b442cfc59f => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:19.502729 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9728d-d9bc-45f7-a924-05505a769d06 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:19.528439 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:19.588660 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:19.628096 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0685c1-7098-4970-9fac-695365a0d857 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:19.657079 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f9728d-d9bc-45f7-a924-05505a769d06 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:19.669079 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8afe0dd-6ef3-4336-b62b-e45ea063e466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:19.669793 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8afe0dd-6ef3-4336-b62b-e45ea063e466 None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:19.670194 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8afe0dd-6ef3-4336-b62b-e45ea063e466 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:19.670528 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8afe0dd-6ef3-4336-b62b-e45ea063e466 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:19.671248 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8afe0dd-6ef3-4336-b62b-e45ea063e466 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:19.672869 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8afe0dd-6ef3-4336-b62b-e45ea063e466 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:19.691563 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 437/866] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Sep 28 15:29:19 2022] POST /identity/v3/services => generated 269 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:29:19.708989 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac8338-8993-4cf3-8bf5-a1dc728533e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:19.710183 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac8338-8993-4cf3-8bf5-a1dc728533e2 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:19.710486 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac8338-8993-4cf3-8bf5-a1dc728533e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:19.710758 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac8338-8993-4cf3-8bf5-a1dc728533e2 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:19.711507 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ac8338-8993-4cf3-8bf5-a1dc728533e2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:19.711869 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ac8338-8993-4cf3-8bf5-a1dc728533e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:19.713381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ac8338-8993-4cf3-8bf5-a1dc728533e2 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:19.736284 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 430/867] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Sep 28 15:29:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:19.742382 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8afe0dd-6ef3-4336-b62b-e45ea063e466 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:19.862400 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8afe0dd-6ef3-4336-b62b-e45ea063e466 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:19.895496 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:19.901815 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a360e4c5-4a1f-4edc-a1b1-95ebad0ea309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:19.902495 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360e4c5-4a1f-4edc-a1b1-95ebad0ea309 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:19.902784 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360e4c5-4a1f-4edc-a1b1-95ebad0ea309 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:19.903230 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a360e4c5-4a1f-4edc-a1b1-95ebad0ea309 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:19.976927 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a360e4c5-4a1f-4edc-a1b1-95ebad0ea309 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:19.978483 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a360e4c5-4a1f-4edc-a1b1-95ebad0ea309 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:20.086247 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 438/868] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:29:19 2022] GET /identity/v3/auth/tokens => generated 1271 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 28 15:29:20.091672 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac8338-8993-4cf3-8bf5-a1dc728533e2 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:20.151567 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:20.191943 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac8338-8993-4cf3-8bf5-a1dc728533e2 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:20.231323 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6baea37-b7a8-4ad6-921f-efdde34ac224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:20.231953 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6baea37-b7a8-4ad6-921f-efdde34ac224 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:20.232227 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6baea37-b7a8-4ad6-921f-efdde34ac224 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:20.232491 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6baea37-b7a8-4ad6-921f-efdde34ac224 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:20.243143 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6baea37-b7a8-4ad6-921f-efdde34ac224 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:20.244579 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6baea37-b7a8-4ad6-921f-efdde34ac224 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:20.260069 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-f6baea37-b7a8-4ad6-921f-efdde34ac224 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:20.260069 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:20.260069 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:20.260069 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:20.260069 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:20.260069 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:20.260069 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:20.260069 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:20.260069 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:20.260069 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:20.260069 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:20.260069 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:20.260069 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:20.260069 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:20.260069 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:20.260069 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:20.261040 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:20.261040 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:20.261040 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:20.261040 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:20.261040 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:20.261040 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:20.261040 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:20.261040 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:20.261040 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:20.261040 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:20.261040 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:20.261040 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:20.261040 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:20.261040 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:20.261040 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:20.261040 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:20.261040 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:20.262439 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:20.262439 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:20.262439 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:20.262439 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 431/869] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:20.274557 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a360e4c5-4a1f-4edc-a1b1-95ebad0ea309 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:20.358376 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:20.434808 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a360e4c5-4a1f-4edc-a1b1-95ebad0ea309 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:20.469619 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6ad893-c85d-4c53-b3e4-c4f9572a2436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:20.470368 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6ad893-c85d-4c53-b3e4-c4f9572a2436 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:20.470728 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6ad893-c85d-4c53-b3e4-c4f9572a2436 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:20.471068 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6ad893-c85d-4c53-b3e4-c4f9572a2436 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:20.471777 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6ad893-c85d-4c53-b3e4-c4f9572a2436 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:20.472147 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6ad893-c85d-4c53-b3e4-c4f9572a2436 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:20.473632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6ad893-c85d-4c53-b3e4-c4f9572a2436 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:20.485729 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 439/870] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:20.495826 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6baea37-b7a8-4ad6-921f-efdde34ac224 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:20.539840 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9943e180-e455-473e-8553-b0f58c3b56d4 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:20.540114 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9943e180-e455-473e-8553-b0f58c3b56d4 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:20.540363 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9943e180-e455-473e-8553-b0f58c3b56d4 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:20.541713 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 440/871] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:20.548690 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884a6c44-7b6d-4888-996e-3719134a5ce2 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:20.549056 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884a6c44-7b6d-4888-996e-3719134a5ce2 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:20.549407 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884a6c44-7b6d-4888-996e-3719134a5ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:20.573816 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-884a6c44-7b6d-4888-996e-3719134a5ce2 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:20.575335 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-884a6c44-7b6d-4888-996e-3719134a5ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:20.626435 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6baea37-b7a8-4ad6-921f-efdde34ac224 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:20.661236 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:20.674948 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b2e02-c840-4ae1-a818-bfa2921c3b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:20.675599 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b2e02-c840-4ae1-a818-bfa2921c3b94 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:20.676266 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b2e02-c840-4ae1-a818-bfa2921c3b94 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:20.676527 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b2e02-c840-4ae1-a818-bfa2921c3b94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:20.686182 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28b2e02-c840-4ae1-a818-bfa2921c3b94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:20.688622 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28b2e02-c840-4ae1-a818-bfa2921c3b94 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:20.708837 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-a28b2e02-c840-4ae1-a818-bfa2921c3b94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:20.708837 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:20.708837 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:20.708837 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:20.708837 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:20.708837 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:20.708837 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:20.708837 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:20.708837 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:20.708837 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:20.708837 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:20.708837 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:20.708837 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:29:20.708837 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:29:20.708837 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:29:20.710492 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:29:20.710492 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:20.710492 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:20.710492 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:20.710492 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:20.710492 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:20.710492 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:20.710492 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:20.710492 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:20.710492 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:20.710492 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:20.710492 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:20.710492 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:20.710492 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:20.710492 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:20.710492 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:20.710492 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:20.711425 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:29:20.711425 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:29:20.711425 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:29:20.711425 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:29:20.711425 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:29:20.711425 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:29:20.711425 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:20.711425 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:20.719603 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 432/872] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:29:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:20.721524 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b2e02-c840-4ae1-a818-bfa2921c3b94 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:20.746043 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 441/873] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:20 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 15:29:20.838310 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b2e02-c840-4ae1-a818-bfa2921c3b94 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:20.882595 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f5765a-b4a8-4ed2-be10-530c31e3d9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:20.883474 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5765a-b4a8-4ed2-be10-530c31e3d9c7 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:20.883938 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5765a-b4a8-4ed2-be10-530c31e3d9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:20.884404 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5765a-b4a8-4ed2-be10-530c31e3d9c7 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:20.885265 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f5765a-b4a8-4ed2-be10-530c31e3d9c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:20.885752 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f5765a-b4a8-4ed2-be10-530c31e3d9c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:20.887367 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f5765a-b4a8-4ed2-be10-530c31e3d9c7 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:20.906018 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 433/874] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 15:29:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:20.906937 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884a6c44-7b6d-4888-996e-3719134a5ce2 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:20.934864 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:21.017888 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884a6c44-7b6d-4888-996e-3719134a5ce2 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:21.056021 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f5765a-b4a8-4ed2-be10-530c31e3d9c7 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:21.060663 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6aef47-7f0c-4a1d-a6f2-e43003395c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:21.061282 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6aef47-7f0c-4a1d-a6f2-e43003395c1b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:21.061559 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6aef47-7f0c-4a1d-a6f2-e43003395c1b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:21.061829 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6aef47-7f0c-4a1d-a6f2-e43003395c1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:21.062601 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6aef47-7f0c-4a1d-a6f2-e43003395c1b None admin] RBAC: Adding query filter params (role.id=342b64fe0cdb4925b23c6cd461a152ac, scope.project.id=3dc6c5f5fcd34922b0d09cd2aaed6cf1, user.id=c81e5fb007614cf2992f5e02e85d6307) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:21.062909 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6aef47-7f0c-4a1d-a6f2-e43003395c1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:21.064340 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6aef47-7f0c-4a1d-a6f2-e43003395c1b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:21.072853 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 442/875] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 15:29:20 2022] GET /identity/v3/role_assignments?user.id=c81e5fb007614cf2992f5e02e85d6307&scope.project.id=3dc6c5f5fcd34922b0d09cd2aaed6cf1&role.id=342b64fe0cdb4925b23c6cd461a152ac => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:21.104259 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:21.180573 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f5765a-b4a8-4ed2-be10-530c31e3d9c7 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:21.217135 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e529b2-9e9b-4439-86ce-be775ac55d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:21.218173 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e529b2-9e9b-4439-86ce-be775ac55d3a None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:21.218668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e529b2-9e9b-4439-86ce-be775ac55d3a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:21.219140 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e529b2-9e9b-4439-86ce-be775ac55d3a None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:21.220013 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e529b2-9e9b-4439-86ce-be775ac55d3a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:21.221674 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e529b2-9e9b-4439-86ce-be775ac55d3a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:21.243396 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 434/876] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Sep 28 15:29:21 2022] POST /identity/v3/services => generated 251 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:29:21.904469 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a88eaf-bd33-4a7a-9616-5556ae66ebb7 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:21.905019 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a88eaf-bd33-4a7a-9616-5556ae66ebb7 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:21.905019 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a88eaf-bd33-4a7a-9616-5556ae66ebb7 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:21.906617 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 443/877] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:21.912391 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690a034-2dad-45bc-a176-8b750a8f0617 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:21.912746 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690a034-2dad-45bc-a176-8b750a8f0617 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:21.913619 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690a034-2dad-45bc-a176-8b750a8f0617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:21.935112 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-a690a034-2dad-45bc-a176-8b750a8f0617 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:21.936323 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a690a034-2dad-45bc-a176-8b750a8f0617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:22.020471 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 435/878] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:21 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 15:29:22.072351 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099f2aa-36cf-4a75-a7a4-e96704ebac99 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:22.072997 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099f2aa-36cf-4a75-a7a4-e96704ebac99 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:22.073481 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099f2aa-36cf-4a75-a7a4-e96704ebac99 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:22.075353 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 444/879] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:22.081251 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5839c0e9-5701-44ed-bee1-706e3ad204db None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:22.081931 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5839c0e9-5701-44ed-bee1-706e3ad204db None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:22.082430 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5839c0e9-5701-44ed-bee1-706e3ad204db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:22.103535 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-5839c0e9-5701-44ed-bee1-706e3ad204db None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:22.105079 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5839c0e9-5701-44ed-bee1-706e3ad204db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:22.120160 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:22.185423 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 436/880] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:22 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:29:22.307802 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e099f2aa-36cf-4a75-a7a4-e96704ebac99 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:22.404362 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:22.421533 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e099f2aa-36cf-4a75-a7a4-e96704ebac99 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:22.457292 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc7a859-77cd-4ed5-95c9-c67a86df47ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:22.457841 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7a859-77cd-4ed5-95c9-c67a86df47ad None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:22.458178 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7a859-77cd-4ed5-95c9-c67a86df47ad None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:22.458446 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7a859-77cd-4ed5-95c9-c67a86df47ad None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:22.459024 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc7a859-77cd-4ed5-95c9-c67a86df47ad None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:22.460414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc7a859-77cd-4ed5-95c9-c67a86df47ad None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:22.469632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-ccc7a859-77cd-4ed5-95c9-c67a86df47ad None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 15:29:22.469632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:22.469632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:29:22.469632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:29:22.469632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:29:22.469632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:29:22.469632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:29:22.469632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:29:22.469632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 15:29:22.469632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:22.469632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:29:22.469632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:22.469632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:22.469632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:22.469632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:22.469632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:22.469632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:22.470694 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:22.470694 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:22.470694 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:22.470694 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:22.470694 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:22.470694 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:22.470694 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:29:22.470694 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:29:22.470694 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:29:22.470694 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:29:22.470694 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:22.470694 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:22.470694 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:22.470694 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:22.470694 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:22.470694 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:22.471925 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:22.471925 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:22.471925 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:22.471925 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:22.471925 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:22.471925 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:22.471925 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:22.471925 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:22.471925 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:22.471925 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:22.471925 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:29:22.471925 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:29:22.471925 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 15:29:22.471925 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:22.471925 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 445/881] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Sep 28 15:29:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:22.477331 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5839c0e9-5701-44ed-bee1-706e3ad204db None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:22.499083 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc7a859-77cd-4ed5-95c9-c67a86df47ad None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:22.578158 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5839c0e9-5701-44ed-bee1-706e3ad204db None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:22.606697 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc7a859-77cd-4ed5-95c9-c67a86df47ad None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:22.612536 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe70575-41e3-4608-be66-b13975676f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:22.613137 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe70575-41e3-4608-be66-b13975676f43 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:22.613414 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe70575-41e3-4608-be66-b13975676f43 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:22.613682 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe70575-41e3-4608-be66-b13975676f43 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:22.614323 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe70575-41e3-4608-be66-b13975676f43 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:22.614621 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe70575-41e3-4608-be66-b13975676f43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:22.616024 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe70575-41e3-4608-be66-b13975676f43 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:22.625470 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 437/882] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Sep 28 15:29:22 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:22.626414 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe70575-41e3-4608-be66-b13975676f43 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:22.641091 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:22.642763 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ae0de8-50c9-4b52-8bae-d2b6a6734a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:22.643393 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae0de8-50c9-4b52-8bae-d2b6a6734a90 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:22.643760 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae0de8-50c9-4b52-8bae-d2b6a6734a90 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:22.644033 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae0de8-50c9-4b52-8bae-d2b6a6734a90 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:22.644612 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ae0de8-50c9-4b52-8bae-d2b6a6734a90 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:22.646158 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ae0de8-50c9-4b52-8bae-d2b6a6734a90 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:22.657476 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-a7ae0de8-50c9-4b52-8bae-d2b6a6734a90 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 28 15:29:22.657476 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:22.657476 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:29:22.657476 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:29:22.657476 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:29:22.657476 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:29:22.657476 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:29:22.657476 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:29:22.657476 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 28 15:29:22.657476 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:22.657476 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:29:22.657476 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:22.657476 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:22.657476 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:22.657476 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:22.657476 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:22.658441 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:22.658441 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:22.658441 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:22.658441 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:22.658441 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:22.658441 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:22.658441 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:22.658441 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:29:22.658441 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:29:22.658441 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:29:22.658441 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:29:22.658441 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:22.658441 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:22.658441 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:22.658441 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:22.658441 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:22.658441 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:22.659540 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:22.659540 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:22.659540 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:22.659540 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:22.659540 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:22.659540 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:22.659540 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:22.659540 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:22.659540 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:22.659540 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:22.659540 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:29:22.659540 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:29:22.659540 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 28 15:29:22.659540 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:22.659540 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 446/883] 10.222.0.52 () {56 vars in 1132 bytes} [Wed Sep 28 15:29:22 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:29:22.673840 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ae0de8-50c9-4b52-8bae-d2b6a6734a90 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:22.694483 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:22.736665 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe70575-41e3-4608-be66-b13975676f43 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:22.770688 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428fbaa2-069f-4e5e-96f4-d4cf8572d859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:22.771284 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428fbaa2-069f-4e5e-96f4-d4cf8572d859 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:22.771561 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428fbaa2-069f-4e5e-96f4-d4cf8572d859 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:22.771830 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428fbaa2-069f-4e5e-96f4-d4cf8572d859 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:22.772983 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428fbaa2-069f-4e5e-96f4-d4cf8572d859 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:22.773282 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428fbaa2-069f-4e5e-96f4-d4cf8572d859 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:22.774681 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428fbaa2-069f-4e5e-96f4-d4cf8572d859 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:22.786150 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 438/884] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Sep 28 15:29:22 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:22.804382 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ae0de8-50c9-4b52-8bae-d2b6a6734a90 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:22.844022 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436ad506-1be9-46d2-8a4f-2696c0bfe72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:22.844926 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ad506-1be9-46d2-8a4f-2696c0bfe72b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:22.845346 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ad506-1be9-46d2-8a4f-2696c0bfe72b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:22.845770 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ad506-1be9-46d2-8a4f-2696c0bfe72b None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:22.846601 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436ad506-1be9-46d2-8a4f-2696c0bfe72b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:22.847054 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436ad506-1be9-46d2-8a4f-2696c0bfe72b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:22.848585 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436ad506-1be9-46d2-8a4f-2696c0bfe72b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:22.859781 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 447/885] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Sep 28 15:29:22 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:22.861503 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428fbaa2-069f-4e5e-96f4-d4cf8572d859 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:22.953610 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:22.978321 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428fbaa2-069f-4e5e-96f4-d4cf8572d859 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:23.021314 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76c0ae8-ef89-458d-9b09-c6cb4a1d9242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:23.021865 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c0ae8-ef89-458d-9b09-c6cb4a1d9242 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:23.022832 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c0ae8-ef89-458d-9b09-c6cb4a1d9242 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:23.023109 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c0ae8-ef89-458d-9b09-c6cb4a1d9242 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:23.023735 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76c0ae8-ef89-458d-9b09-c6cb4a1d9242 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:23.024030 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76c0ae8-ef89-458d-9b09-c6cb4a1d9242 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:23.025385 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76c0ae8-ef89-458d-9b09-c6cb4a1d9242 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:23.030703 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 439/886] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Sep 28 15:29:22 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:23.036739 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436ad506-1be9-46d2-8a4f-2696c0bfe72b None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:23.123390 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:23.146467 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436ad506-1be9-46d2-8a4f-2696c0bfe72b None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:23.182793 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13b3087-39df-4fb3-a508-3a33cebc79b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:23.183445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b3087-39df-4fb3-a508-3a33cebc79b8 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:23.183724 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b3087-39df-4fb3-a508-3a33cebc79b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:23.184001 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b3087-39df-4fb3-a508-3a33cebc79b8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:23.184686 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13b3087-39df-4fb3-a508-3a33cebc79b8 None admin] RBAC: Adding query filter params (interface=public, service_id=7470415d901245c098c843237da8989e, region_id=RegionOne) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:23.184992 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13b3087-39df-4fb3-a508-3a33cebc79b8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:23.186449 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13b3087-39df-4fb3-a508-3a33cebc79b8 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:23.192249 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 448/887] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Sep 28 15:29:23 2022] GET /identity/v3/endpoints?service_id=7470415d901245c098c843237da8989e&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:23.200171 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76c0ae8-ef89-458d-9b09-c6cb4a1d9242 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:23.312968 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76c0ae8-ef89-458d-9b09-c6cb4a1d9242 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:23.351570 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0b485e-cc10-45dc-8bc0-32a8c8e8eda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:23.352578 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b485e-cc10-45dc-8bc0-32a8c8e8eda8 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:23.353095 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b485e-cc10-45dc-8bc0-32a8c8e8eda8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:23.353629 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b485e-cc10-45dc-8bc0-32a8c8e8eda8 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:23.354466 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0b485e-cc10-45dc-8bc0-32a8c8e8eda8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:23.356058 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0b485e-cc10-45dc-8bc0-32a8c8e8eda8 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:23.366325 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 440/888] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Sep 28 15:29:23 2022] GET /identity/v3/services => generated 1268 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:29:23.446816 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ccf76b-ca41-4c40-8ea4-ce8a4baf7180 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:23.447452 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ccf76b-ca41-4c40-8ea4-ce8a4baf7180 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:23.447922 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ccf76b-ca41-4c40-8ea4-ce8a4baf7180 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:23.449770 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 449/889] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:23.455637 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56352205-7903-4c9e-b1e4-1b15b40ac15f None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:23.456281 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56352205-7903-4c9e-b1e4-1b15b40ac15f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:23.456773 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56352205-7903-4c9e-b1e4-1b15b40ac15f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:23.464844 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:23.477875 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-56352205-7903-4c9e-b1e4-1b15b40ac15f None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:23.479898 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-56352205-7903-4c9e-b1e4-1b15b40ac15f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:23.509654 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8999e6e9-acd9-48d9-9b8c-7eac1a4b17c3 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:23.510425 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8999e6e9-acd9-48d9-9b8c-7eac1a4b17c3 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:23.510968 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8999e6e9-acd9-48d9-9b8c-7eac1a4b17c3 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:23.512910 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 450/890] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:23.518900 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8664add-871d-40bd-93de-72dca6d0c1be None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:23.519528 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8664add-871d-40bd-93de-72dca6d0c1be None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:23.520067 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8664add-871d-40bd-93de-72dca6d0c1be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:23.539972 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-a8664add-871d-40bd-93de-72dca6d0c1be None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:23.541638 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a8664add-871d-40bd-93de-72dca6d0c1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:23.557383 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 441/891] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:23 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:29:23.622292 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 451/892] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:23 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 15:29:23.836168 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56352205-7903-4c9e-b1e4-1b15b40ac15f None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:23.909911 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8664add-871d-40bd-93de-72dca6d0c1be None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:23.944322 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56352205-7903-4c9e-b1e4-1b15b40ac15f None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:23.996301 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6f4086-5d1d-460a-a4c0-17e97b07732b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:23.997173 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f4086-5d1d-460a-a4c0-17e97b07732b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:23.997708 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f4086-5d1d-460a-a4c0-17e97b07732b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:23.998957 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:23.999094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f4086-5d1d-460a-a4c0-17e97b07732b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:23.999094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6f4086-5d1d-460a-a4c0-17e97b07732b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:24.002106 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6f4086-5d1d-460a-a4c0-17e97b07732b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:24.004555 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:24.015501 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-aa6f4086-5d1d-460a-a4c0-17e97b07732b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 28 15:29:24.015501 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:24.015501 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:29:24.015501 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:29:24.015501 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:29:24.015501 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:29:24.015501 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:29:24.015501 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:29:24.015501 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 28 15:29:24.015501 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:24.015501 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:29:24.015501 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:24.015501 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:24.015501 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:24.015501 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:24.015501 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:24.015501 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:24.016529 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:24.016529 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:24.016529 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:24.016529 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:24.016529 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:24.016529 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:24.016529 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:29:24.016529 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:29:24.016529 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:29:24.016529 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:29:24.016529 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:24.016529 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:24.016529 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:24.016529 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:24.016529 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:24.016529 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:24.017388 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:24.017388 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:24.017388 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:24.017388 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:24.017388 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:24.017388 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:24.017388 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:24.017388 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:24.017388 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:24.017388 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:24.017388 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:29:24.017388 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:29:24.017388 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 28 15:29:24.017388 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:24.025352 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 442/893] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 15:29:23 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:29:24.027354 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6f4086-5d1d-460a-a4c0-17e97b07732b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:24.044680 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8664add-871d-40bd-93de-72dca6d0c1be None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:24.090273 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f900415-bea4-462c-a842-02b65e8f1bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:24.091250 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f900415-bea4-462c-a842-02b65e8f1bb9 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:24.091731 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f900415-bea4-462c-a842-02b65e8f1bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:24.092204 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f900415-bea4-462c-a842-02b65e8f1bb9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:24.103978 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f900415-bea4-462c-a842-02b65e8f1bb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:24.109642 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-5f900415-bea4-462c-a842-02b65e8f1bb9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 15:29:24.109642 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:24.109642 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:24.109642 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:24.109642 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:24.109642 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:24.109642 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:24.109642 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:24.109642 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:24.109642 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:24.109642 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:24.109642 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:24.109642 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:24.109642 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:24.109642 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:24.109642 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:24.110784 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:24.110784 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:24.110784 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:24.110784 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:24.110784 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:24.110784 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:24.110784 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:24.110784 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:24.110784 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:24.110784 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:24.110784 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:24.110784 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:24.110784 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:24.110784 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:24.110784 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:24.110784 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:24.112630 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:24.112630 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:24.112630 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:24.112630 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:24.112630 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:24.112630 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:24.112630 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:24.112630 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:24.112630 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:24.112630 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:24.112630 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 15:29:24.112630 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:24.115280 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 452/894] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Sep 28 15:29:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:29:24.120940 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f900415-bea4-462c-a842-02b65e8f1bb9 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:24.154034 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6f4086-5d1d-460a-a4c0-17e97b07732b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:24.190316 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fdeb0e-45dd-4d3d-8d71-520c4afc69f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:24.190991 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fdeb0e-45dd-4d3d-8d71-520c4afc69f7 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:24.191210 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fdeb0e-45dd-4d3d-8d71-520c4afc69f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:24.191472 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fdeb0e-45dd-4d3d-8d71-520c4afc69f7 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:24.192091 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fdeb0e-45dd-4d3d-8d71-520c4afc69f7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:24.192393 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fdeb0e-45dd-4d3d-8d71-520c4afc69f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:24.193781 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fdeb0e-45dd-4d3d-8d71-520c4afc69f7 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:24.199154 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 443/895] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 15:29:24 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:24.206296 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fdeb0e-45dd-4d3d-8d71-520c4afc69f7 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:24.237964 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f900415-bea4-462c-a842-02b65e8f1bb9 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:24.276363 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:24.279308 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:24.285601 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a53ccf7-9a5c-419b-b824-eae7f52f9de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:24.286575 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a53ccf7-9a5c-419b-b824-eae7f52f9de0 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:24.287120 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a53ccf7-9a5c-419b-b824-eae7f52f9de0 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:24.287664 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a53ccf7-9a5c-419b-b824-eae7f52f9de0 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:24.288829 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a53ccf7-9a5c-419b-b824-eae7f52f9de0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:24.293277 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a53ccf7-9a5c-419b-b824-eae7f52f9de0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:24.295715 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a53ccf7-9a5c-419b-b824-eae7f52f9de0 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:24.306676 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 453/896] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Sep 28 15:29:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:24.313254 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a53ccf7-9a5c-419b-b824-eae7f52f9de0 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:24.324086 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fdeb0e-45dd-4d3d-8d71-520c4afc69f7 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:24.365209 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2336f20-4a60-4990-9483-a6240b5efe56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:24.366853 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2336f20-4a60-4990-9483-a6240b5efe56 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:24.367143 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2336f20-4a60-4990-9483-a6240b5efe56 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:24.367421 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2336f20-4a60-4990-9483-a6240b5efe56 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:24.368049 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2336f20-4a60-4990-9483-a6240b5efe56 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:24.368349 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2336f20-4a60-4990-9483-a6240b5efe56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:24.370106 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2336f20-4a60-4990-9483-a6240b5efe56 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:24.377928 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 444/897] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 15:29:24 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:24.383107 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2336f20-4a60-4990-9483-a6240b5efe56 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:24.449692 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a53ccf7-9a5c-419b-b824-eae7f52f9de0 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:24.494651 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf647f-170e-4334-8786-947e1942533d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:24.496363 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf647f-170e-4334-8786-947e1942533d None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:24.496973 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf647f-170e-4334-8786-947e1942533d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:24.497555 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf647f-170e-4334-8786-947e1942533d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:24.505694 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2336f20-4a60-4990-9483-a6240b5efe56 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:24.512638 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf647f-170e-4334-8786-947e1942533d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:24.514553 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf647f-170e-4334-8786-947e1942533d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:24.532665 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-e9bf647f-170e-4334-8786-947e1942533d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:24.532665 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:24.532665 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:24.532665 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:24.532665 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:24.532665 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:24.532665 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:24.532665 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:24.532665 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:24.532665 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:24.532665 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:24.532665 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:24.532665 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:24.532665 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:24.532665 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:24.532665 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:24.533660 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:24.533660 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:24.533660 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:24.533660 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:24.533660 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:24.533660 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:24.533660 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:24.533660 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:24.533660 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:24.533660 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:24.533660 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:24.533660 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:24.533660 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:24.533660 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:24.533660 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:24.533660 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:24.533660 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:24.534705 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:24.534705 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:24.534705 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:24.539165 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 454/898] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:24.540369 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf647f-170e-4334-8786-947e1942533d None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:24.541189 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:24.550961 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ef6769-0f96-4590-9daf-3abb4af132ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:24.551989 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ef6769-0f96-4590-9daf-3abb4af132ec None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:24.552447 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ef6769-0f96-4590-9daf-3abb4af132ec None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:24.552952 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ef6769-0f96-4590-9daf-3abb4af132ec None admin] PATH_INFO: `/v3/endpoints` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:24.555454 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ef6769-0f96-4590-9daf-3abb4af132ec None admin] RBAC: Adding query filter params (interface=public, service_id=71179bbc654d4d9086424df701b272e0, region_id=RegionOne) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:24.555963 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ef6769-0f96-4590-9daf-3abb4af132ec None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:24.557732 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ef6769-0f96-4590-9daf-3abb4af132ec None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:24.563899 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 445/899] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Sep 28 15:29:24 2022] GET /identity/v3/endpoints?service_id=71179bbc654d4d9086424df701b272e0&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:24.571386 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ef6769-0f96-4590-9daf-3abb4af132ec None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:24.605217 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:24.681574 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf647f-170e-4334-8786-947e1942533d None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:24.682224 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ef6769-0f96-4590-9daf-3abb4af132ec None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:24.719906 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e55c946-4690-479e-bc76-e4034e9bff73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:24.720604 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e55c946-4690-479e-bc76-e4034e9bff73 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:24.720885 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e55c946-4690-479e-bc76-e4034e9bff73 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:24.721154 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e55c946-4690-479e-bc76-e4034e9bff73 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:24.721978 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e55c946-4690-479e-bc76-e4034e9bff73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:24.723341 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e55c946-4690-479e-bc76-e4034e9bff73 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:24.728387 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c2b63-2f6e-45ad-9b66-801474f1e2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:24.729246 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c2b63-2f6e-45ad-9b66-801474f1e2e6 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:24.729717 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c2b63-2f6e-45ad-9b66-801474f1e2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:24.730021 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 446/900] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Sep 28 15:29:24 2022] GET /identity/v3/services => generated 1268 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:29:24.730349 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c2b63-2f6e-45ad-9b66-801474f1e2e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:24.731089 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17c2b63-2f6e-45ad-9b66-801474f1e2e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:24.731402 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17c2b63-2f6e-45ad-9b66-801474f1e2e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:24.732788 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17c2b63-2f6e-45ad-9b66-801474f1e2e6 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:24.747633 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 455/901] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:24.753091 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e55c946-4690-479e-bc76-e4034e9bff73 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:24.820598 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:24.871770 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e55c946-4690-479e-bc76-e4034e9bff73 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:24.914779 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ae9e0c-b3be-459c-8470-367c801e13c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:24.915443 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae9e0c-b3be-459c-8470-367c801e13c0 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:24.915726 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae9e0c-b3be-459c-8470-367c801e13c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:24.916000 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae9e0c-b3be-459c-8470-367c801e13c0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:24.924560 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ae9e0c-b3be-459c-8470-367c801e13c0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:24.926537 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ae9e0c-b3be-459c-8470-367c801e13c0 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:24.938183 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-a7ae9e0c-b3be-459c-8470-367c801e13c0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 15:29:24.938183 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:24.938183 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:24.938183 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:24.938183 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:24.938183 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:24.938183 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:24.938183 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:24.938183 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:24.938183 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:24.938183 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:24.938183 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:24.938183 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:29:24.938183 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:29:24.938183 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:29:24.938183 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:29:24.939121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:24.939121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:24.939121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:29:24.939121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:24.939121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:29:24.939121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:24.939121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:24.939121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:24.939121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:24.939121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:24.939121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:24.939121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:24.939121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:24.939121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:24.939121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:24.939121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:24.939121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:24.939121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:24.940235 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:24.940235 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:24.940235 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:29:24.940235 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:29:24.940235 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:29:24.940235 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:29:24.940235 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:29:24.940235 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:29:24.940235 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 15:29:24.940235 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:24.940235 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 447/902] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Sep 28 15:29:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:24.946990 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c2b63-2f6e-45ad-9b66-801474f1e2e6 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:25.009376 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674d867-47a7-4612-84b1-7176ba665777 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:25.009664 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674d867-47a7-4612-84b1-7176ba665777 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:25.009985 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674d867-47a7-4612-84b1-7176ba665777 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:25.011312 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 448/903] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:25.016812 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c522e-6cd8-4adb-895e-76c7a5929407 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:25.017089 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c522e-6cd8-4adb-895e-76c7a5929407 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:25.017359 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c522e-6cd8-4adb-895e-76c7a5929407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:25.036850 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d24c522e-6cd8-4adb-895e-76c7a5929407 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:25.040835 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d24c522e-6cd8-4adb-895e-76c7a5929407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:25.070690 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c2b63-2f6e-45ad-9b66-801474f1e2e6 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:25.081011 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:25.117840 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3552b33-c241-4112-902c-bf0b9ce67e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:25.118796 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3552b33-c241-4112-902c-bf0b9ce67e8a None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:25.119069 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3552b33-c241-4112-902c-bf0b9ce67e8a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:25.119344 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3552b33-c241-4112-902c-bf0b9ce67e8a None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:25.120318 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3552b33-c241-4112-902c-bf0b9ce67e8a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:25.120621 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3552b33-c241-4112-902c-bf0b9ce67e8a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:25.122726 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3552b33-c241-4112-902c-bf0b9ce67e8a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:25.126755 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 449/904] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:25 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 15:29:25.147717 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 456/905] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Sep 28 15:29:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:25.153452 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c522e-6cd8-4adb-895e-76c7a5929407 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:25.198291 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:25.253174 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c522e-6cd8-4adb-895e-76c7a5929407 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:25.305904 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99bcf6f-de40-4d12-ad0b-8350b012cb85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:25.305904 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99bcf6f-de40-4d12-ad0b-8350b012cb85 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:25.305904 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99bcf6f-de40-4d12-ad0b-8350b012cb85 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:25.305904 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99bcf6f-de40-4d12-ad0b-8350b012cb85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:25.376058 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99bcf6f-de40-4d12-ad0b-8350b012cb85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:25.377531 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99bcf6f-de40-4d12-ad0b-8350b012cb85 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:25.421806 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3552b33-c241-4112-902c-bf0b9ce67e8a None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:25.425473 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:25.494159 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 450/906] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:29:25 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 15:29:25.501451 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99bcf6f-de40-4d12-ad0b-8350b012cb85 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:25.548155 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3552b33-c241-4112-902c-bf0b9ce67e8a None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:25.585302 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21725cf6-7db9-416c-bb80-3088174ed038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:25.585946 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21725cf6-7db9-416c-bb80-3088174ed038 None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:25.586224 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21725cf6-7db9-416c-bb80-3088174ed038 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:25.586490 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21725cf6-7db9-416c-bb80-3088174ed038 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:25.587149 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21725cf6-7db9-416c-bb80-3088174ed038 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:25.588507 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21725cf6-7db9-416c-bb80-3088174ed038 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:25.603442 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 457/907] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Sep 28 15:29:25 2022] POST /identity/v3/services => generated 241 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:29:25.604938 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b6b1d-94f0-400e-9b34-649fbcf997bf None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:25.605427 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b6b1d-94f0-400e-9b34-649fbcf997bf None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:25.605944 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b6b1d-94f0-400e-9b34-649fbcf997bf None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:25.611456 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 458/908] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:25.613021 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976007b-0a13-4c2a-bde3-843d5e484d35 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:25.613302 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976007b-0a13-4c2a-bde3-843d5e484d35 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:25.613571 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976007b-0a13-4c2a-bde3-843d5e484d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:25.617913 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99bcf6f-de40-4d12-ad0b-8350b012cb85 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:25.634227 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-b976007b-0a13-4c2a-bde3-843d5e484d35 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:25.635877 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b976007b-0a13-4c2a-bde3-843d5e484d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:25.658018 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:25.674259 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd30258-a1fd-4334-8ecb-3366055bfaa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:25.675612 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd30258-a1fd-4334-8ecb-3366055bfaa7 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:25.676127 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd30258-a1fd-4334-8ecb-3366055bfaa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:25.676668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd30258-a1fd-4334-8ecb-3366055bfaa7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:25.688527 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:25.691655 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd30258-a1fd-4334-8ecb-3366055bfaa7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:25.694414 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd30258-a1fd-4334-8ecb-3366055bfaa7 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:25.721193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-5fd30258-a1fd-4334-8ecb-3366055bfaa7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:25.721193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:25.721193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:25.721193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:25.721193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:25.721193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:25.721193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:25.721193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:25.721193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:25.721193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:25.721193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:25.721193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:25.721193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:25.721193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:25.721193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:25.721193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:25.722313 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:25.722313 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:25.722313 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:25.722313 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:25.722313 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:25.722313 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:25.722313 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:25.722313 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:25.722313 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:25.722313 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:25.722313 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:25.722313 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:25.722313 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:25.722313 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:25.722313 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:25.722313 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:25.722313 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:25.723207 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:25.723207 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:25.723207 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:25.723207 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 451/909] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:25.734111 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd30258-a1fd-4334-8ecb-3366055bfaa7 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:25.734566 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 459/910] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 15:29:25.845111 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd30258-a1fd-4334-8ecb-3366055bfaa7 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:25.880911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4272e48e-8f14-4194-beca-3155379447ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:25.882166 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4272e48e-8f14-4194-beca-3155379447ba None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:25.882442 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4272e48e-8f14-4194-beca-3155379447ba None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:25.882710 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4272e48e-8f14-4194-beca-3155379447ba None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:25.883337 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4272e48e-8f14-4194-beca-3155379447ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:25.883631 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4272e48e-8f14-4194-beca-3155379447ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:25.884992 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4272e48e-8f14-4194-beca-3155379447ba None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:25.900910 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 452/911] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:25.903111 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b976007b-0a13-4c2a-bde3-843d5e484d35 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:26.010169 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b976007b-0a13-4c2a-bde3-843d5e484d35 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:26.014801 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4272e48e-8f14-4194-beca-3155379447ba None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:26.047550 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4318e0-f895-4451-b6ff-89ba5fd2f091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:26.048621 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4318e0-f895-4451-b6ff-89ba5fd2f091 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:26.049103 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4318e0-f895-4451-b6ff-89ba5fd2f091 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:26.049654 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4318e0-f895-4451-b6ff-89ba5fd2f091 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:26.057576 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4318e0-f895-4451-b6ff-89ba5fd2f091 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:26.059401 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4318e0-f895-4451-b6ff-89ba5fd2f091 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:26.071856 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-3b4318e0-f895-4451-b6ff-89ba5fd2f091 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:26.071856 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:26.071856 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:26.071856 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:26.071856 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:26.071856 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:26.071856 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:26.071856 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:26.071856 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:26.071856 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:26.071856 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:26.071856 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:26.071856 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:29:26.071856 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:29:26.071856 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:29:26.073221 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:29:26.073221 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:26.073221 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:26.073221 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:26.073221 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:26.073221 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:26.073221 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:26.073221 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:26.073221 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:26.073221 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:26.073221 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:26.073221 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:26.073221 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:26.073221 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:26.073221 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:26.073221 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:26.073221 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:26.074160 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:29:26.074160 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:29:26.074160 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:29:26.074160 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:29:26.074160 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:29:26.074160 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:29:26.074160 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:26.074160 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:26.076613 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 460/912] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:29:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:26.083007 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:26.083934 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4318e0-f895-4451-b6ff-89ba5fd2f091 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:26.106974 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:26.144890 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4272e48e-8f14-4194-beca-3155379447ba None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:26.183751 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f177c6d-514c-4f80-a6d8-00148b85e96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:26.184385 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f177c6d-514c-4f80-a6d8-00148b85e96a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:26.184666 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f177c6d-514c-4f80-a6d8-00148b85e96a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:26.184943 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f177c6d-514c-4f80-a6d8-00148b85e96a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:26.185588 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f177c6d-514c-4f80-a6d8-00148b85e96a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:26.187067 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f177c6d-514c-4f80-a6d8-00148b85e96a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:26.196870 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-1f177c6d-514c-4f80-a6d8-00148b85e96a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 28 15:29:26.196870 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:26.196870 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:29:26.196870 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:29:26.196870 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:29:26.196870 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:29:26.196870 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:29:26.196870 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:29:26.196870 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 28 15:29:26.196870 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:26.196870 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:29:26.196870 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:26.196870 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:26.196870 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:26.196870 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:26.196870 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:26.197824 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:26.197824 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:26.197824 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:26.197824 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:26.197824 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:26.197824 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:26.197824 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:26.197824 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:29:26.197824 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:29:26.197824 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:29:26.197824 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:29:26.197824 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:26.197824 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:26.197824 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:26.197824 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:26.197824 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:26.197824 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:26.203744 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:26.203744 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:26.203744 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:26.203744 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:26.203744 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:26.203744 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:26.203744 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:26.203744 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:26.203744 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:26.203744 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:26.203744 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:29:26.203744 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:29:26.203744 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 28 15:29:26.203744 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:26.203744 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 453/913] 10.222.0.52 () {56 vars in 1132 bytes} [Wed Sep 28 15:29:26 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:26.206222 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f177c6d-514c-4f80-a6d8-00148b85e96a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:26.225645 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4318e0-f895-4451-b6ff-89ba5fd2f091 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:26.274607 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73493dd3-1a29-4f2f-8d5d-0a1e08df1410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:26.275259 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73493dd3-1a29-4f2f-8d5d-0a1e08df1410 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:26.275538 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73493dd3-1a29-4f2f-8d5d-0a1e08df1410 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:26.275811 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73493dd3-1a29-4f2f-8d5d-0a1e08df1410 None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:26.276563 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73493dd3-1a29-4f2f-8d5d-0a1e08df1410 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:26.276872 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73493dd3-1a29-4f2f-8d5d-0a1e08df1410 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:26.278833 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73493dd3-1a29-4f2f-8d5d-0a1e08df1410 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:26.303886 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 461/914] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 15:29:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:26.305509 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73493dd3-1a29-4f2f-8d5d-0a1e08df1410 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:26.340386 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f177c6d-514c-4f80-a6d8-00148b85e96a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:26.357486 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:26.376751 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0825f4c6-cf61-4968-9144-704f31dcdedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:26.377362 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0825f4c6-cf61-4968-9144-704f31dcdedd None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:26.377643 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0825f4c6-cf61-4968-9144-704f31dcdedd None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:26.377944 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0825f4c6-cf61-4968-9144-704f31dcdedd None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:26.378572 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0825f4c6-cf61-4968-9144-704f31dcdedd None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:26.378875 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0825f4c6-cf61-4968-9144-704f31dcdedd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:26.380508 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0825f4c6-cf61-4968-9144-704f31dcdedd None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:26.390681 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 454/915] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Sep 28 15:29:26 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:26.394854 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0825f4c6-cf61-4968-9144-704f31dcdedd None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:26.396830 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:26.434995 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73493dd3-1a29-4f2f-8d5d-0a1e08df1410 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:26.478614 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034127eb-f01c-48cb-82a5-433620fbbb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:26.479424 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034127eb-f01c-48cb-82a5-433620fbbb31 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:26.479974 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034127eb-f01c-48cb-82a5-433620fbbb31 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:26.480349 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034127eb-f01c-48cb-82a5-433620fbbb31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:26.481488 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034127eb-f01c-48cb-82a5-433620fbbb31 None admin] RBAC: Adding query filter params (role.id=769ffe2b49154917b28e916692c53d5f, scope.project.id=3dc6c5f5fcd34922b0d09cd2aaed6cf1, user.id=c81e5fb007614cf2992f5e02e85d6307) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:26.482184 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034127eb-f01c-48cb-82a5-433620fbbb31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:26.483695 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034127eb-f01c-48cb-82a5-433620fbbb31 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:26.497530 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 462/916] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 15:29:26 2022] GET /identity/v3/role_assignments?user.id=c81e5fb007614cf2992f5e02e85d6307&scope.project.id=3dc6c5f5fcd34922b0d09cd2aaed6cf1&role.id=769ffe2b49154917b28e916692c53d5f => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:26.527333 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0825f4c6-cf61-4968-9144-704f31dcdedd None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:26.566198 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400a1766-02f4-4df4-bb7b-6b66c88ad426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:26.566820 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400a1766-02f4-4df4-bb7b-6b66c88ad426 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:26.567102 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400a1766-02f4-4df4-bb7b-6b66c88ad426 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:26.567378 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400a1766-02f4-4df4-bb7b-6b66c88ad426 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:26.568015 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400a1766-02f4-4df4-bb7b-6b66c88ad426 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:26.568319 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400a1766-02f4-4df4-bb7b-6b66c88ad426 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:26.570370 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400a1766-02f4-4df4-bb7b-6b66c88ad426 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:26.576428 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 455/917] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Sep 28 15:29:26 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:26.583338 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034127eb-f01c-48cb-82a5-433620fbbb31 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:26.701390 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034127eb-f01c-48cb-82a5-433620fbbb31 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:26.743241 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba34d1b-5512-409b-b4db-1724f2d24ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:26.744210 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba34d1b-5512-409b-b4db-1724f2d24ba6 None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:26.744743 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba34d1b-5512-409b-b4db-1724f2d24ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:26.745212 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba34d1b-5512-409b-b4db-1724f2d24ba6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:26.746134 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba34d1b-5512-409b-b4db-1724f2d24ba6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:26.747712 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba34d1b-5512-409b-b4db-1724f2d24ba6 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:26.757289 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:26.781940 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 463/918] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:29:26 2022] POST /identity/v3/endpoints => generated 347 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:29:27.046050 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f516ee-876d-42f1-9fa9-23deaa11a639 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:27.046554 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f516ee-876d-42f1-9fa9-23deaa11a639 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:27.046554 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f516ee-876d-42f1-9fa9-23deaa11a639 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:27.048226 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 456/919] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:27.053491 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b41a78c-a145-46bb-b526-53bd224529bd None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:27.053776 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b41a78c-a145-46bb-b526-53bd224529bd None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:27.054090 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b41a78c-a145-46bb-b526-53bd224529bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:27.073272 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-5b41a78c-a145-46bb-b526-53bd224529bd None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:27.074514 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-5b41a78c-a145-46bb-b526-53bd224529bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:27.152368 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 464/920] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:29:27.419706 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f516ee-876d-42f1-9fa9-23deaa11a639 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:27.469062 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:27.526398 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f516ee-876d-42f1-9fa9-23deaa11a639 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:27.561586 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c436eeb-8690-424e-b910-757719ecc564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:27.562248 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c436eeb-8690-424e-b910-757719ecc564 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:27.562536 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c436eeb-8690-424e-b910-757719ecc564 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:27.562859 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c436eeb-8690-424e-b910-757719ecc564 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:27.563444 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c436eeb-8690-424e-b910-757719ecc564 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:27.565019 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c436eeb-8690-424e-b910-757719ecc564 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:27.574226 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-1c436eeb-8690-424e-b910-757719ecc564 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 28 15:29:27.574226 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:27.574226 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:29:27.574226 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:29:27.574226 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:29:27.574226 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:29:27.574226 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:29:27.574226 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:29:27.574226 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 28 15:29:27.574226 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:27.574226 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:29:27.574226 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:27.574226 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:27.574226 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:27.574226 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:27.574226 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:27.574226 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:27.575265 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:27.575265 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:27.575265 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:27.575265 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:27.575265 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:27.575265 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:27.575265 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:29:27.575265 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:29:27.575265 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:29:27.575265 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:29:27.575265 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:27.575265 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:27.575265 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:27.575265 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:27.575265 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:27.575265 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:27.576651 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:27.576651 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:27.576651 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:27.576651 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:27.576651 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:27.576651 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:27.576651 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:27.576651 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:27.576651 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:27.576651 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:27.576651 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:29:27.576651 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:29:27.576651 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 28 15:29:27.576651 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:27.576651 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 457/921] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 15:29:27 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:27.581152 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b41a78c-a145-46bb-b526-53bd224529bd None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:27.686695 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b41a78c-a145-46bb-b526-53bd224529bd None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:27.721046 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65ef30-6681-49e5-aedf-e1296eba1b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:27.722033 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65ef30-6681-49e5-aedf-e1296eba1b7f None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:27.722608 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65ef30-6681-49e5-aedf-e1296eba1b7f None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:27.723075 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65ef30-6681-49e5-aedf-e1296eba1b7f None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:27.723900 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e65ef30-6681-49e5-aedf-e1296eba1b7f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:27.724449 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e65ef30-6681-49e5-aedf-e1296eba1b7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:27.726548 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e65ef30-6681-49e5-aedf-e1296eba1b7f None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:27.730205 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:27.732328 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 465/922] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 15:29:27 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:27.738670 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c436eeb-8690-424e-b910-757719ecc564 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:27.857261 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c436eeb-8690-424e-b910-757719ecc564 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:27.868750 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:27.895280 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfaf91f-8e85-4e9d-9bb9-e25cc562466a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:27.895884 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfaf91f-8e85-4e9d-9bb9-e25cc562466a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:27.896200 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfaf91f-8e85-4e9d-9bb9-e25cc562466a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:27.896481 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfaf91f-8e85-4e9d-9bb9-e25cc562466a None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:27.897146 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfaf91f-8e85-4e9d-9bb9-e25cc562466a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:27.897448 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfaf91f-8e85-4e9d-9bb9-e25cc562466a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:27.898848 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfaf91f-8e85-4e9d-9bb9-e25cc562466a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:27.904249 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 458/923] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Sep 28 15:29:27 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:27.909492 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65ef30-6681-49e5-aedf-e1296eba1b7f None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:27.955211 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ab732-e417-4b6c-a791-cfeeba279bbd None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:27.955497 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ab732-e417-4b6c-a791-cfeeba279bbd None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:27.955681 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3ab732-e417-4b6c-a791-cfeeba279bbd None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:27.957327 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 459/924] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:29:27.965892 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ddab4-c5bc-4be2-9c01-b6bafb4bfcbc None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:27.965892 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ddab4-c5bc-4be2-9c01-b6bafb4bfcbc None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:27.965892 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ddab4-c5bc-4be2-9c01-b6bafb4bfcbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:27.987118 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-aa7ddab4-c5bc-4be2-9c01-b6bafb4bfcbc None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:27.989493 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-aa7ddab4-c5bc-4be2-9c01-b6bafb4bfcbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:28.022674 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65ef30-6681-49e5-aedf-e1296eba1b7f None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:28.060507 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a98b758-2c81-4435-ad8f-5d79a2496d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:28.061112 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98b758-2c81-4435-ad8f-5d79a2496d64 None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:28.061407 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98b758-2c81-4435-ad8f-5d79a2496d64 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:28.061680 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98b758-2c81-4435-ad8f-5d79a2496d64 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:28.062470 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a98b758-2c81-4435-ad8f-5d79a2496d64 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:28.063936 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a98b758-2c81-4435-ad8f-5d79a2496d64 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:28.074269 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 460/925] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:29:28.090444 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 466/926] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:29:27 2022] POST /identity/v3/endpoints => generated 346 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:29:28.350804 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7ddab4-c5bc-4be2-9c01-b6bafb4bfcbc None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:28.472747 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7ddab4-c5bc-4be2-9c01-b6bafb4bfcbc None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:28.475303 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:28.518984 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefc0387-15cd-4245-b7dd-d9d1e7aecf8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:28.520104 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc0387-15cd-4245-b7dd-d9d1e7aecf8a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:28.520588 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc0387-15cd-4245-b7dd-d9d1e7aecf8a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:28.521069 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc0387-15cd-4245-b7dd-d9d1e7aecf8a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:28.522457 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefc0387-15cd-4245-b7dd-d9d1e7aecf8a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:28.524048 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefc0387-15cd-4245-b7dd-d9d1e7aecf8a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:28.533026 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-aefc0387-15cd-4245-b7dd-d9d1e7aecf8a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 15:29:28.533026 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:28.533026 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:29:28.533026 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:29:28.533026 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:29:28.533026 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:29:28.533026 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:29:28.533026 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:29:28.533026 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 15:29:28.533026 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:28.533026 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:29:28.533026 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:28.533026 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:28.533026 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:28.533026 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:28.533026 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:28.533026 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:28.534041 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:28.534041 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:28.534041 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:28.534041 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:28.534041 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:28.534041 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:28.534041 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:29:28.534041 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:29:28.534041 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:29:28.534041 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:29:28.534041 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:28.534041 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:28.534041 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:28.534041 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:28.534041 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:28.534041 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:28.534888 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:28.534888 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:28.534888 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:28.534888 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:28.534888 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:28.534888 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:28.534888 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:28.534888 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:28.534888 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:28.534888 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:28.534888 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:29:28.534888 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:29:28.534888 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 15:29:28.534888 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:28.538262 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 461/927] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Sep 28 15:29:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:28.544513 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a98b758-2c81-4435-ad8f-5d79a2496d64 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:28.590182 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:28.652075 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a98b758-2c81-4435-ad8f-5d79a2496d64 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:28.686950 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c665ce-b309-43b5-807a-3f1739f613f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:28.687909 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c665ce-b309-43b5-807a-3f1739f613f7 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:28.688426 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c665ce-b309-43b5-807a-3f1739f613f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:28.688907 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c665ce-b309-43b5-807a-3f1739f613f7 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:28.689748 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c665ce-b309-43b5-807a-3f1739f613f7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:28.690304 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c665ce-b309-43b5-807a-3f1739f613f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:28.691864 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c665ce-b309-43b5-807a-3f1739f613f7 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:28.698041 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 467/928] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Sep 28 15:29:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:28.704127 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefc0387-15cd-4245-b7dd-d9d1e7aecf8a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:28.816792 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefc0387-15cd-4245-b7dd-d9d1e7aecf8a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:28.852011 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd53c6a-f79a-4482-a39e-8b83f4e50f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:28.852544 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd53c6a-f79a-4482-a39e-8b83f4e50f9e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:28.852826 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd53c6a-f79a-4482-a39e-8b83f4e50f9e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:28.853099 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd53c6a-f79a-4482-a39e-8b83f4e50f9e None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:28.853723 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd53c6a-f79a-4482-a39e-8b83f4e50f9e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:28.854057 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd53c6a-f79a-4482-a39e-8b83f4e50f9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:28.855427 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd53c6a-f79a-4482-a39e-8b83f4e50f9e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:28.860843 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 462/929] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Sep 28 15:29:28 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:28.864369 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab9b2cc-2d4a-4c2e-9367-e240e589aa13 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:28.864934 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab9b2cc-2d4a-4c2e-9367-e240e589aa13 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:28.866095 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab9b2cc-2d4a-4c2e-9367-e240e589aa13 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:28.868006 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 468/930] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:28.870873 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd53c6a-f79a-4482-a39e-8b83f4e50f9e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:28.874002 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b6b8e9-7bcc-42df-b351-aca8b882b65d None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:28.874301 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b6b8e9-7bcc-42df-b351-aca8b882b65d None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:28.874574 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b6b8e9-7bcc-42df-b351-aca8b882b65d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:28.899121 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-c2b6b8e9-7bcc-42df-b351-aca8b882b65d None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:28.900702 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c2b6b8e9-7bcc-42df-b351-aca8b882b65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:28.902739 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:28.986732 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd53c6a-f79a-4482-a39e-8b83f4e50f9e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:29.001987 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 469/931] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 15:29:29.028427 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1ac24f-1ac0-402c-b400-35694b80d0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:29.029384 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ac24f-1ac0-402c-b400-35694b80d0dc None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:29.029933 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ac24f-1ac0-402c-b400-35694b80d0dc None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:29.030281 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1ac24f-1ac0-402c-b400-35694b80d0dc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:29.031023 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1ac24f-1ac0-402c-b400-35694b80d0dc None admin] RBAC: Adding query filter params (interface=public, service_id=a1c25e40b06c4486a573c015c523c4b8, region_id=RegionOne) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:29.031349 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1ac24f-1ac0-402c-b400-35694b80d0dc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:29.032747 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1ac24f-1ac0-402c-b400-35694b80d0dc None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:29.038964 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 463/932] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Sep 28 15:29:28 2022] GET /identity/v3/endpoints?service_id=a1c25e40b06c4486a573c015c523c4b8&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:29.044366 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b6b8e9-7bcc-42df-b351-aca8b882b65d None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:29.084976 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3da7e-e3cf-4ddb-8cc9-da1c2160d57e None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:29.085840 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3da7e-e3cf-4ddb-8cc9-da1c2160d57e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:29.086139 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3da7e-e3cf-4ddb-8cc9-da1c2160d57e None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:29.087618 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 464/933] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:29.091423 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:29.094968 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8337acf4-77c8-49da-8e99-1633ec3fe0de None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:29.095511 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8337acf4-77c8-49da-8e99-1633ec3fe0de None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:29.095998 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8337acf4-77c8-49da-8e99-1633ec3fe0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:29.118797 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-8337acf4-77c8-49da-8e99-1633ec3fe0de None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:29.121455 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8337acf4-77c8-49da-8e99-1633ec3fe0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:29.170862 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b6b8e9-7bcc-42df-b351-aca8b882b65d None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:29.206366 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91ecd18-8f9e-4f4a-a9ae-58620f097e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:29.207292 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ecd18-8f9e-4f4a-a9ae-58620f097e66 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:29.209886 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 465/934] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 15:29:29.214109 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ecd18-8f9e-4f4a-a9ae-58620f097e66 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:29.214476 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ecd18-8f9e-4f4a-a9ae-58620f097e66 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:29.215164 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91ecd18-8f9e-4f4a-a9ae-58620f097e66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:29.216595 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91ecd18-8f9e-4f4a-a9ae-58620f097e66 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:29.223395 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 470/935] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Sep 28 15:29:29 2022] GET /identity/v3/services => generated 1497 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:29:29.305656 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8337acf4-77c8-49da-8e99-1633ec3fe0de None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:29.329797 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:29.421803 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8337acf4-77c8-49da-8e99-1633ec3fe0de None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:29.458029 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d200ce1b-08d4-4315-a9aa-c8149e70e2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:29.458671 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d200ce1b-08d4-4315-a9aa-c8149e70e2ce None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:29.458949 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d200ce1b-08d4-4315-a9aa-c8149e70e2ce None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:29.459346 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d200ce1b-08d4-4315-a9aa-c8149e70e2ce None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:29.469900 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d200ce1b-08d4-4315-a9aa-c8149e70e2ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:29.474947 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-d200ce1b-08d4-4315-a9aa-c8149e70e2ce None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 15:29:29.474947 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:29.474947 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:29.474947 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:29.474947 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:29.474947 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:29.474947 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:29.474947 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:29.474947 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:29.474947 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:29.474947 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:29.474947 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:29.474947 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:29.474947 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:29.474947 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:29.474947 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:29.475896 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:29.475896 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:29.475896 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:29.475896 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:29.475896 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:29.475896 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:29.475896 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:29.475896 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:29.475896 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:29.475896 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:29.475896 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:29.475896 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:29.475896 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:29.475896 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:29.475896 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:29.475896 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:29.476746 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:29.476746 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:29.476746 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:29.476746 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:29.476746 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:29.476746 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:29.476746 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:29.476746 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:29.476746 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:29.476746 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:29.476746 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 15:29:29.476746 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:29.476746 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 466/936] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Sep 28 15:29:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:29.482058 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91ecd18-8f9e-4f4a-a9ae-58620f097e66 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:29.495192 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d200ce1b-08d4-4315-a9aa-c8149e70e2ce None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:29.587112 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:29.597363 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:29.601832 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91ecd18-8f9e-4f4a-a9ae-58620f097e66 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:29.606916 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d200ce1b-08d4-4315-a9aa-c8149e70e2ce None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:29.642535 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174db737-9be8-4945-8bdf-150ce5d8bd99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:29.643321 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237457b8-7bae-4d1c-8330-1c6a04b4def0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:29.643590 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174db737-9be8-4945-8bdf-150ce5d8bd99 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:29.644011 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174db737-9be8-4945-8bdf-150ce5d8bd99 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:29.644125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237457b8-7bae-4d1c-8330-1c6a04b4def0 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:29.644480 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237457b8-7bae-4d1c-8330-1c6a04b4def0 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:29.644686 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174db737-9be8-4945-8bdf-150ce5d8bd99 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:29.644879 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237457b8-7bae-4d1c-8330-1c6a04b4def0 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:29.645525 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237457b8-7bae-4d1c-8330-1c6a04b4def0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:29.645794 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174db737-9be8-4945-8bdf-150ce5d8bd99 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:29.646278 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174db737-9be8-4945-8bdf-150ce5d8bd99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:29.647145 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237457b8-7bae-4d1c-8330-1c6a04b4def0 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:29.648034 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174db737-9be8-4945-8bdf-150ce5d8bd99 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:29.656304 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-237457b8-7bae-4d1c-8330-1c6a04b4def0 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 28 15:29:29.656304 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:29.656304 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:29:29.656304 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:29:29.656304 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:29:29.656304 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:29:29.656304 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:29:29.656304 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:29:29.656304 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 28 15:29:29.656304 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:29.656304 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:29:29.656304 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:29.656304 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:29.656304 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:29.656304 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:29.656304 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:29.656304 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:29.657544 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:29.657544 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:29.657544 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:29.657544 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:29.657544 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:29.657544 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:29.657544 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:29:29.657544 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:29:29.657544 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:29:29.657544 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:29:29.657544 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:29.657544 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:29.657544 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:29.657544 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:29.657544 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:29.657544 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:29.658621 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:29.658621 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:29.658621 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:29.658621 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:29.658621 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:29.658621 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:29.658621 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:29.658621 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:29.658621 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:29.658621 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:29.658621 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:29:29.658621 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:29:29.658621 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 28 15:29:29.658621 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:29.660576 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 471/938] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Sep 28 15:29:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:29.665083 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 467/938] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:29:29 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:29:29.666224 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174db737-9be8-4945-8bdf-150ce5d8bd99 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:29.666439 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237457b8-7bae-4d1c-8330-1c6a04b4def0 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:29.782455 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237457b8-7bae-4d1c-8330-1c6a04b4def0 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:29.799934 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174db737-9be8-4945-8bdf-150ce5d8bd99 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:29.828177 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ab33b-89b8-44b2-aee8-2233fff4582e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:29.829321 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ab33b-89b8-44b2-aee8-2233fff4582e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:29.830093 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ab33b-89b8-44b2-aee8-2233fff4582e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:29.830974 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ab33b-89b8-44b2-aee8-2233fff4582e None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:29.832134 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030ab33b-89b8-44b2-aee8-2233fff4582e None admin] RBAC: Adding query filter params (name=compute) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:29.832635 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030ab33b-89b8-44b2-aee8-2233fff4582e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:29.834870 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030ab33b-89b8-44b2-aee8-2233fff4582e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:29.836168 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123816f0-88be-4221-9973-84d47e75d0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:29.836791 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123816f0-88be-4221-9973-84d47e75d0dd None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:29.837064 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123816f0-88be-4221-9973-84d47e75d0dd None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:29.837331 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123816f0-88be-4221-9973-84d47e75d0dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:29.841862 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 468/939] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Sep 28 15:29:29 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:29.847439 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ab33b-89b8-44b2-aee8-2233fff4582e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:29.854555 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123816f0-88be-4221-9973-84d47e75d0dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:29.856101 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123816f0-88be-4221-9973-84d47e75d0dd None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:29.861967 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:29.872697 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-123816f0-88be-4221-9973-84d47e75d0dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:29.872697 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:29.872697 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:29.872697 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:29.872697 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:29.872697 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:29.872697 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:29.872697 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:29.872697 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:29.872697 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:29.872697 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:29.872697 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:29.872697 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:29.872697 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:29.872697 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:29.872697 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:29.873706 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:29.873706 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:29.873706 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:29.873706 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:29.873706 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:29.873706 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:29.873706 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:29.873706 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:29.873706 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:29.873706 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:29.873706 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:29.873706 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:29.873706 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:29.873706 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:29.873706 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:29.873706 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:29.873706 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:29.874661 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:29.874661 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:29.874661 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:29.874661 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 472/940] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:29:29.879341 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123816f0-88be-4221-9973-84d47e75d0dd None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:29.906258 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:29.979897 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ab33b-89b8-44b2-aee8-2233fff4582e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:29.997976 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123816f0-88be-4221-9973-84d47e75d0dd None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:30.018362 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece2f62e-9764-45ee-b886-7f9ca23bdf97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:30.019099 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2f62e-9764-45ee-b886-7f9ca23bdf97 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:30.019495 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2f62e-9764-45ee-b886-7f9ca23bdf97 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:30.019829 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2f62e-9764-45ee-b886-7f9ca23bdf97 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:30.020567 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece2f62e-9764-45ee-b886-7f9ca23bdf97 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:30.020932 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece2f62e-9764-45ee-b886-7f9ca23bdf97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:30.022949 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece2f62e-9764-45ee-b886-7f9ca23bdf97 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:30.034292 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 469/941] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Sep 28 15:29:29 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:30.045202 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229ff677-eb29-4a96-a668-57cfb623b051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:30.046216 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ff677-eb29-4a96-a668-57cfb623b051 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:30.046680 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ff677-eb29-4a96-a668-57cfb623b051 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:30.047149 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ff677-eb29-4a96-a668-57cfb623b051 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:30.048000 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229ff677-eb29-4a96-a668-57cfb623b051 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:30.048511 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229ff677-eb29-4a96-a668-57cfb623b051 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:30.050170 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229ff677-eb29-4a96-a668-57cfb623b051 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:30.064489 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 473/942] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:30.070889 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece2f62e-9764-45ee-b886-7f9ca23bdf97 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:30.149814 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:30.187006 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece2f62e-9764-45ee-b886-7f9ca23bdf97 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:30.229221 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e853a0ec-f28a-42d2-b205-9bb807e49771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:30.230375 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853a0ec-f28a-42d2-b205-9bb807e49771 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:30.230654 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853a0ec-f28a-42d2-b205-9bb807e49771 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:30.230936 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853a0ec-f28a-42d2-b205-9bb807e49771 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:30.239706 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e853a0ec-f28a-42d2-b205-9bb807e49771 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:30.241761 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e853a0ec-f28a-42d2-b205-9bb807e49771 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:30.252930 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-e853a0ec-f28a-42d2-b205-9bb807e49771 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 15:29:30.252930 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:30.252930 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:30.252930 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:30.252930 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:30.252930 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:30.252930 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:30.252930 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:30.252930 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:30.252930 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:30.252930 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:30.252930 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:30.252930 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:29:30.252930 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:29:30.252930 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:29:30.252930 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:29:30.253869 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:30.253869 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:30.253869 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:29:30.253869 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:30.253869 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:29:30.253869 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:30.253869 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:30.253869 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:30.253869 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:30.253869 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:30.253869 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:30.253869 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:30.253869 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:30.253869 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:30.253869 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:30.253869 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:30.253869 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:30.253869 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:30.255332 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:30.255332 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:30.255332 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:29:30.255332 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:29:30.255332 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:29:30.255332 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:29:30.255332 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:29:30.255332 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:29:30.255332 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 15:29:30.255332 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:30.255332 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 470/943] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Sep 28 15:29:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:30.262701 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229ff677-eb29-4a96-a668-57cfb623b051 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:30.344312 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:30.375476 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229ff677-eb29-4a96-a668-57cfb623b051 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:30.410488 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02da76c5-2c4e-4af0-b279-a20f7230f3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:30.411099 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02da76c5-2c4e-4af0-b279-a20f7230f3ed None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:30.411377 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02da76c5-2c4e-4af0-b279-a20f7230f3ed None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:30.411648 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02da76c5-2c4e-4af0-b279-a20f7230f3ed None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:30.413013 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02da76c5-2c4e-4af0-b279-a20f7230f3ed None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:30.413315 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02da76c5-2c4e-4af0-b279-a20f7230f3ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:30.414782 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02da76c5-2c4e-4af0-b279-a20f7230f3ed None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:30.424838 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95aa55a-14ba-48d5-a707-5c7804629387 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:30.425431 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95aa55a-14ba-48d5-a707-5c7804629387 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:30.425929 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95aa55a-14ba-48d5-a707-5c7804629387 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:30.427689 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 471/944] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:30.433526 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6d82c-c477-44d7-81f9-7f22528bedb8 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:30.433877 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 474/945] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Sep 28 15:29:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:30.434345 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6d82c-c477-44d7-81f9-7f22528bedb8 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:30.434829 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6d82c-c477-44d7-81f9-7f22528bedb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:30.439363 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02da76c5-2c4e-4af0-b279-a20f7230f3ed None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:30.454884 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-6ab6d82c-c477-44d7-81f9-7f22528bedb8 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:30.456405 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6ab6d82c-c477-44d7-81f9-7f22528bedb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:30.537875 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 472/946] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 15:29:30.544982 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02da76c5-2c4e-4af0-b279-a20f7230f3ed None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:30.574219 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:30.581643 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6a8c5b-f570-4058-b029-c3603a4400d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:30.582877 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a8c5b-f570-4058-b029-c3603a4400d7 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:30.583157 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a8c5b-f570-4058-b029-c3603a4400d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:30.583543 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a8c5b-f570-4058-b029-c3603a4400d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:30.595013 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6a8c5b-f570-4058-b029-c3603a4400d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:30.596464 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6a8c5b-f570-4058-b029-c3603a4400d7 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:30.612052 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-3c6a8c5b-f570-4058-b029-c3603a4400d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:30.612052 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:30.612052 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:30.612052 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:30.612052 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:30.612052 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:30.612052 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:30.612052 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:30.612052 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:30.612052 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:30.612052 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:30.612052 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:30.612052 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:30.612052 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:30.612052 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:30.612052 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:30.613003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:30.613003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:30.613003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:30.613003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:30.613003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:30.613003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:30.613003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:30.613003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:30.613003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:30.613003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:30.613003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:30.613003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:30.613003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:30.613003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:30.613003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:30.613003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:30.613003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:30.613896 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:30.613896 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:30.613896 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:30.613896 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 475/947] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:29:30.623734 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab6d82c-c477-44d7-81f9-7f22528bedb8 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:30.687142 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:30.726613 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab6d82c-c477-44d7-81f9-7f22528bedb8 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:30.762056 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e51ebd-25ae-46e3-ba56-c245d9338095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:30.762666 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e51ebd-25ae-46e3-ba56-c245d9338095 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:30.762949 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e51ebd-25ae-46e3-ba56-c245d9338095 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:30.763221 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e51ebd-25ae-46e3-ba56-c245d9338095 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:30.763916 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e51ebd-25ae-46e3-ba56-c245d9338095 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:30.764218 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e51ebd-25ae-46e3-ba56-c245d9338095 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:30.765591 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e51ebd-25ae-46e3-ba56-c245d9338095 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:30.777372 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 473/948] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:30.782973 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6a8c5b-f570-4058-b029-c3603a4400d7 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:30.821658 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e51ebd-25ae-46e3-ba56-c245d9338095 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:30.909900 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6a8c5b-f570-4058-b029-c3603a4400d7 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:30.930331 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e51ebd-25ae-46e3-ba56-c245d9338095 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:30.958144 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:30.961207 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a5d28a-5ef2-4cba-ba7d-4e96cac6fec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:30.962051 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a5d28a-5ef2-4cba-ba7d-4e96cac6fec4 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:30.962646 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a5d28a-5ef2-4cba-ba7d-4e96cac6fec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:30.963246 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a5d28a-5ef2-4cba-ba7d-4e96cac6fec4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:30.971449 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff495c46-1cee-4a29-b4a2-f522e9f7e1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:30.972340 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff495c46-1cee-4a29-b4a2-f522e9f7e1a3 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:30.972711 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a5d28a-5ef2-4cba-ba7d-4e96cac6fec4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:30.972830 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff495c46-1cee-4a29-b4a2-f522e9f7e1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:30.973217 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff495c46-1cee-4a29-b4a2-f522e9f7e1a3 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:30.973989 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff495c46-1cee-4a29-b4a2-f522e9f7e1a3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:30.974224 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a5d28a-5ef2-4cba-ba7d-4e96cac6fec4 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:30.976908 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff495c46-1cee-4a29-b4a2-f522e9f7e1a3 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:30.979719 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: WARNING keystone.server.flask.common [None req-ff495c46-1cee-4a29-b4a2-f522e9f7e1a3 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 28 15:29:30.987894 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-39a5d28a-5ef2-4cba-ba7d-4e96cac6fec4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:30.987894 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:30.987894 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:30.987894 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:30.987894 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:30.987894 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:30.987894 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:30.987894 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:30.987894 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:30.987894 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:30.987894 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:30.987894 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:30.987894 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:29:30.987894 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:29:30.987894 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:29:30.988954 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:29:30.988954 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:30.988954 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:30.988954 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:30.988954 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:30.988954 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:30.988954 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:30.988954 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:30.988954 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:30.988954 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:30.988954 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:30.988954 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:30.988954 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:30.988954 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:30.988954 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:30.988954 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:30.988954 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:30.990149 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:29:30.990149 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:29:30.990149 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:29:30.990149 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:29:30.990149 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:29:30.990149 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:29:30.990149 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:30.990149 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:30.994560 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 476/949] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:29:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:30.995508 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a5d28a-5ef2-4cba-ba7d-4e96cac6fec4 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:31.006756 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.sql.core [None req-ff495c46-1cee-4a29-b4a2-f522e9f7e1a3 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 28 15:29:31.006756 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 28 15:29:31.006756 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [parameters: {'id': 'dd1d388650bc40cf9c56afef5d90d6d8', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 28 15:29:31.006756 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81605) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 28 15:29:31.014408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-ff495c46-1cee-4a29-b4a2-f522e9f7e1a3 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id dd1d388650bc40cf9c56afef5d90d6d8.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id dd1d388650bc40cf9c56afef5d90d6d8. Sep 28 15:29:31.014408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:31.014408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 28 15:29:31.014408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 28 15:29:31.014408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 28 15:29:31.014408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 28 15:29:31.014408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 28 15:29:31.014408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application result = self._query(query) Sep 28 15:29:31.014408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 28 15:29:31.014408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application conn.query(q) Sep 28 15:29:31.014408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 28 15:29:31.014408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 28 15:29:31.014408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 28 15:29:31.015368 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application result.read() Sep 28 15:29:31.015368 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 28 15:29:31.015368 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 28 15:29:31.015368 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 28 15:29:31.015368 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 28 15:29:31.015368 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 28 15:29:31.015368 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 28 15:29:31.015368 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 28 15:29:31.015368 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 28 15:29:31.015368 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 28 15:29:31.015368 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:31.015368 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 28 15:29:31.015368 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:31.015368 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:31.015368 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 28 15:29:31.015368 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 28 15:29:31.015368 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 28 15:29:31.015368 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 28 15:29:31.017058 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 28 15:29:31.017058 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application next(self.gen) Sep 28 15:29:31.017058 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 28 15:29:31.017058 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application yield resource Sep 28 15:29:31.017058 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 28 15:29:31.017058 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application next(self.gen) Sep 28 15:29:31.017058 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 28 15:29:31.017058 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self.session.rollback() Sep 28 15:29:31.017058 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 28 15:29:31.017058 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self.force_reraise() Sep 28 15:29:31.017058 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 28 15:29:31.017058 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise self.value Sep 28 15:29:31.017058 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 28 15:29:31.017058 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 28 15:29:31.017058 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 28 15:29:31.017058 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application session.commit() Sep 28 15:29:31.017058 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 28 15:29:31.018811 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 28 15:29:31.018811 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 28 15:29:31.018811 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._prepare_impl() Sep 28 15:29:31.018811 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 28 15:29:31.018811 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self.session.flush() Sep 28 15:29:31.018811 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 28 15:29:31.018811 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._flush(objects) Sep 28 15:29:31.018811 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 28 15:29:31.018811 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 28 15:29:31.018811 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 28 15:29:31.018811 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application compat.raise_( Sep 28 15:29:31.018811 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 28 15:29:31.018811 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception Sep 28 15:29:31.018811 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 28 15:29:31.018811 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application flush_context.execute() Sep 28 15:29:31.018811 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 28 15:29:31.018811 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rec.execute(self) Sep 28 15:29:31.020364 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 28 15:29:31.020364 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 28 15:29:31.020364 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 28 15:29:31.020364 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 28 15:29:31.020364 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 28 15:29:31.020364 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 28 15:29:31.020364 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 28 15:29:31.020364 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 28 15:29:31.020364 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 28 15:29:31.020364 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 28 15:29:31.020364 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 28 15:29:31.020364 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 28 15:29:31.020364 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 28 15:29:31.020364 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 28 15:29:31.020364 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 28 15:29:31.030543 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 28 15:29:31.030543 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 28 15:29:31.030543 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception Sep 28 15:29:31.030543 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 28 15:29:31.030543 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 28 15:29:31.030543 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 28 15:29:31.030543 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 28 15:29:31.030543 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 28 15:29:31.030543 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application result = self._query(query) Sep 28 15:29:31.030543 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 28 15:29:31.030543 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application conn.query(q) Sep 28 15:29:31.030543 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 28 15:29:31.030543 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 28 15:29:31.030543 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 28 15:29:31.030543 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application result.read() Sep 28 15:29:31.030543 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 28 15:29:31.030543 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 28 15:29:31.032518 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 28 15:29:31.032518 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 28 15:29:31.032518 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 28 15:29:31.032518 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 28 15:29:31.032518 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 28 15:29:31.032518 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 28 15:29:31.032518 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 28 15:29:31.032518 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 28 15:29:31.032518 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [parameters: {'id': 'dd1d388650bc40cf9c56afef5d90d6d8', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 28 15:29:31.032518 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 28 15:29:31.032518 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:31.032518 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:29:31.032518 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:31.032518 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:31.032518 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 28 15:29:31.034212 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 28 15:29:31.034212 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 28 15:29:31.034212 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 28 15:29:31.034212 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 28 15:29:31.034212 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:31.034212 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:29:31.034212 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:31.034212 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:31.034212 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:31.034212 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:31.034212 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:31.034212 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:31.034212 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:31.034212 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:31.034212 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:31.034212 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:31.034212 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:31.036116 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:31.036116 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 28 15:29:31.036116 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 28 15:29:31.036116 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:31.036116 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:31.036116 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 28 15:29:31.036116 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 28 15:29:31.036116 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id dd1d388650bc40cf9c56afef5d90d6d8. Sep 28 15:29:31.036116 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:31.036116 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 474/950] 10.222.0.52 () {60 vars in 1140 bytes} [Wed Sep 28 15:29:30 2022] POST /identity/v3/projects => generated 262 bytes in 202 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:29:31.036116 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff495c46-1cee-4a29-b4a2-f522e9f7e1a3 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:31.052352 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:31.116865 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a5d28a-5ef2-4cba-ba7d-4e96cac6fec4 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:31.138473 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff495c46-1cee-4a29-b4a2-f522e9f7e1a3 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:31.154836 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8944c3c1-4d34-43e1-aa31-f19662fa4f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:31.155458 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8944c3c1-4d34-43e1-aa31-f19662fa4f88 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:31.155829 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8944c3c1-4d34-43e1-aa31-f19662fa4f88 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:31.156110 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8944c3c1-4d34-43e1-aa31-f19662fa4f88 None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:31.156904 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8944c3c1-4d34-43e1-aa31-f19662fa4f88 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:31.157216 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8944c3c1-4d34-43e1-aa31-f19662fa4f88 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:31.158692 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8944c3c1-4d34-43e1-aa31-f19662fa4f88 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:31.175164 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3957860e-21b7-4199-9800-ebed61f38bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:31.175795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957860e-21b7-4199-9800-ebed61f38bf5 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:31.176072 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957860e-21b7-4199-9800-ebed61f38bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:31.176349 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957860e-21b7-4199-9800-ebed61f38bf5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:31.176768 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 477/951] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 15:29:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:31.177588 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8944c3c1-4d34-43e1-aa31-f19662fa4f88 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:31.183851 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3957860e-21b7-4199-9800-ebed61f38bf5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:31.185701 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3957860e-21b7-4199-9800-ebed61f38bf5 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:31.197702 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-3957860e-21b7-4199-9800-ebed61f38bf5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:31.197702 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:31.197702 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:31.197702 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:31.197702 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:31.197702 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:31.197702 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:31.197702 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:31.197702 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:31.197702 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:31.197702 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:31.197702 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:31.197702 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:29:31.197702 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:29:31.197702 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:29:31.199763 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:29:31.199763 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:31.199763 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:31.199763 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:31.199763 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:31.199763 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:31.199763 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:31.199763 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:31.199763 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:31.199763 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:31.199763 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:31.199763 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:31.199763 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:31.199763 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:31.199763 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:31.199763 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:31.199763 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:31.202141 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:29:31.202141 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:29:31.202141 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:29:31.202141 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:29:31.202141 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:29:31.202141 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:29:31.202141 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:31.202141 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:31.209899 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 475/952] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:29:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:31.209899 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3957860e-21b7-4199-9800-ebed61f38bf5 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:31.245340 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:31.289911 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8944c3c1-4d34-43e1-aa31-f19662fa4f88 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:31.292221 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:31.311058 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3957860e-21b7-4199-9800-ebed61f38bf5 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:31.330097 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bf1171-0f6f-4c1a-aa6a-d9c3e04a07b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:31.330962 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf1171-0f6f-4c1a-aa6a-d9c3e04a07b6 None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:31.331384 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf1171-0f6f-4c1a-aa6a-d9c3e04a07b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:31.331824 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf1171-0f6f-4c1a-aa6a-d9c3e04a07b6 None admin] PATH_INFO: `/v3/projects/3dc6c5f5fcd34922b0d09cd2aaed6cf1/users/c81e5fb007614cf2992f5e02e85d6307/roles/769ffe2b49154917b28e916692c53d5f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:31.334967 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bf1171-0f6f-4c1a-aa6a-d9c3e04a07b6 None admin] RBAC: Authorizing `identity:create_grant(project_id=3dc6c5f5fcd34922b0d09cd2aaed6cf1, user_id=c81e5fb007614cf2992f5e02e85d6307, role_id=769ffe2b49154917b28e916692c53d5f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:31.336542 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bf1171-0f6f-4c1a-aa6a-d9c3e04a07b6 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:31.351994 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e361a740-146d-4d20-a863-7405b541df53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:31.352858 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361a740-146d-4d20-a863-7405b541df53 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:31.353542 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361a740-146d-4d20-a863-7405b541df53 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:31.353791 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361a740-146d-4d20-a863-7405b541df53 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:31.354706 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e361a740-146d-4d20-a863-7405b541df53 None admin] RBAC: Adding query filter params (name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:31.355107 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e361a740-146d-4d20-a863-7405b541df53 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:31.356944 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e361a740-146d-4d20-a863-7405b541df53 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:31.365231 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 478/953] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 15:29:31 2022] PUT /identity/v3/projects/3dc6c5f5fcd34922b0d09cd2aaed6cf1/users/c81e5fb007614cf2992f5e02e85d6307/roles/769ffe2b49154917b28e916692c53d5f => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:29:31.383160 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 476/954] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Sep 28 15:29:31 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:31.491858 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb70b89-ae72-46c2-901d-ac1c3ada0c11 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:31.492175 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb70b89-ae72-46c2-901d-ac1c3ada0c11 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:31.492448 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb70b89-ae72-46c2-901d-ac1c3ada0c11 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:31.494186 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 479/955] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:31.500357 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7803be3-52ff-45dc-b9bb-d4a7090b22fc None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:31.500646 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7803be3-52ff-45dc-b9bb-d4a7090b22fc None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:31.500963 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7803be3-52ff-45dc-b9bb-d4a7090b22fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:31.522209 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-e7803be3-52ff-45dc-b9bb-d4a7090b22fc None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:31.523823 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e7803be3-52ff-45dc-b9bb-d4a7090b22fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:31.598591 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 477/956] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:29:31.873955 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb70b89-ae72-46c2-901d-ac1c3ada0c11 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:31.993481 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb70b89-ae72-46c2-901d-ac1c3ada0c11 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:32.030424 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f31d980-2778-41f5-9052-8ce2a368e563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:32.031438 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f31d980-2778-41f5-9052-8ce2a368e563 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:32.031939 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f31d980-2778-41f5-9052-8ce2a368e563 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:32.032410 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f31d980-2778-41f5-9052-8ce2a368e563 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:32.033183 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f31d980-2778-41f5-9052-8ce2a368e563 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:32.034760 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f31d980-2778-41f5-9052-8ce2a368e563 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:32.043983 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-1f31d980-2778-41f5-9052-8ce2a368e563 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 15:29:32.043983 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:32.043983 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:29:32.043983 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:29:32.043983 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:29:32.043983 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:29:32.043983 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:29:32.043983 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:29:32.043983 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 15:29:32.043983 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:32.043983 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:29:32.043983 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:32.043983 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:32.043983 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:32.043983 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:32.043983 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:32.043983 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:32.045085 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:32.045085 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:32.045085 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:32.045085 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:32.045085 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:32.045085 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:32.045085 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:29:32.045085 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:29:32.045085 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:29:32.045085 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:29:32.045085 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:32.045085 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:32.045085 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:32.045085 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:32.045085 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:32.045085 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:32.046037 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:32.046037 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:32.046037 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:32.046037 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:32.046037 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:32.046037 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:32.046037 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:32.046037 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:32.046037 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:32.046037 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:32.046037 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:29:32.046037 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:29:32.046037 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 15:29:32.046037 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:32.048812 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 480/957] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Sep 28 15:29:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:29:32.054644 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7803be3-52ff-45dc-b9bb-d4a7090b22fc None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:32.083006 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:32.158537 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7803be3-52ff-45dc-b9bb-d4a7090b22fc None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:32.193037 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc121b-a1bc-44e3-a1fd-06d7f2400db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:32.194312 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc121b-a1bc-44e3-a1fd-06d7f2400db6 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:32.194580 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc121b-a1bc-44e3-a1fd-06d7f2400db6 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:32.194840 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc121b-a1bc-44e3-a1fd-06d7f2400db6 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:32.195525 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dc121b-a1bc-44e3-a1fd-06d7f2400db6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:32.195834 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dc121b-a1bc-44e3-a1fd-06d7f2400db6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:32.197210 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dc121b-a1bc-44e3-a1fd-06d7f2400db6 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:32.202660 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 478/958] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Sep 28 15:29:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:32.208205 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f31d980-2778-41f5-9052-8ce2a368e563 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:32.237674 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:32.331290 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f31d980-2778-41f5-9052-8ce2a368e563 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:32.371116 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574d3fed-6d28-48ac-a0fd-7d77c1cbcd7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:32.372169 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d3fed-6d28-48ac-a0fd-7d77c1cbcd7e None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:32.372656 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d3fed-6d28-48ac-a0fd-7d77c1cbcd7e None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:32.373134 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d3fed-6d28-48ac-a0fd-7d77c1cbcd7e None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:32.374048 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574d3fed-6d28-48ac-a0fd-7d77c1cbcd7e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:32.374570 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574d3fed-6d28-48ac-a0fd-7d77c1cbcd7e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:32.376210 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574d3fed-6d28-48ac-a0fd-7d77c1cbcd7e None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:32.383571 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 481/959] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Sep 28 15:29:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:32.389878 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc121b-a1bc-44e3-a1fd-06d7f2400db6 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:32.408507 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd1f628-da40-42d3-bfac-e4ee1791c9b4 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:32.409070 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd1f628-da40-42d3-bfac-e4ee1791c9b4 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:32.409495 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd1f628-da40-42d3-bfac-e4ee1791c9b4 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:32.411264 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 482/960] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:29:32.417613 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c1a8a-93fb-4248-a31c-043240144293 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:32.418198 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c1a8a-93fb-4248-a31c-043240144293 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:32.418487 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c1a8a-93fb-4248-a31c-043240144293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:32.445175 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-883c1a8a-93fb-4248-a31c-043240144293 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:32.446414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-883c1a8a-93fb-4248-a31c-043240144293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:32.488062 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:32.499615 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc121b-a1bc-44e3-a1fd-06d7f2400db6 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:32.535430 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5fab86-a0a1-4e33-bf69-41a993065fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:32.536286 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5fab86-a0a1-4e33-bf69-41a993065fbd None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:32.536740 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5fab86-a0a1-4e33-bf69-41a993065fbd None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:32.536989 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 483/961] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 15:29:32.537411 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5fab86-a0a1-4e33-bf69-41a993065fbd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:32.544196 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5fab86-a0a1-4e33-bf69-41a993065fbd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:32.544196 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5fab86-a0a1-4e33-bf69-41a993065fbd None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:32.570062 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 479/962] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:29:32 2022] POST /identity/v3/endpoints => generated 349 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:29:32.811366 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883c1a8a-93fb-4248-a31c-043240144293 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:32.835522 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:32.917586 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883c1a8a-93fb-4248-a31c-043240144293 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:32.962043 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e290832d-8eea-4340-afa0-001710041a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:32.962724 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e290832d-8eea-4340-afa0-001710041a13 None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:32.962997 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e290832d-8eea-4340-afa0-001710041a13 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:32.963259 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e290832d-8eea-4340-afa0-001710041a13 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:32.963891 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e290832d-8eea-4340-afa0-001710041a13 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:32.965233 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e290832d-8eea-4340-afa0-001710041a13 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:32.982369 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 484/963] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Sep 28 15:29:32 2022] POST /identity/v3/services => generated 242 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:29:33.738368 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159aa91f-a6e7-4069-a3d1-64a72489e682 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:33.738892 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159aa91f-a6e7-4069-a3d1-64a72489e682 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:33.738892 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159aa91f-a6e7-4069-a3d1-64a72489e682 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:33.741119 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 480/964] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:33.746347 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8f5d3-453f-466f-bd65-eaa007d33239 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:33.746776 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8f5d3-453f-466f-bd65-eaa007d33239 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:33.746955 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8f5d3-453f-466f-bd65-eaa007d33239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:33.759486 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c814d-04c7-44a1-89bd-e01637dc73c6 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:33.759486 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c814d-04c7-44a1-89bd-e01637dc73c6 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:33.759486 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c814d-04c7-44a1-89bd-e01637dc73c6 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:33.760151 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 481/965] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:29:33.765320 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2697eb-75a4-4ecc-894e-ebb69e7f7018 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:33.765457 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-dbf8f5d3-453f-466f-bd65-eaa007d33239 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:33.765989 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2697eb-75a4-4ecc-894e-ebb69e7f7018 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:33.766493 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2697eb-75a4-4ecc-894e-ebb69e7f7018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:33.766610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-dbf8f5d3-453f-466f-bd65-eaa007d33239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:33.791396 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-ff2697eb-75a4-4ecc-894e-ebb69e7f7018 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:33.793245 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ff2697eb-75a4-4ecc-894e-ebb69e7f7018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:33.852014 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 485/966] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:33 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:29:33.876348 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 482/967] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:33 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 15:29:34.128074 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf8f5d3-453f-466f-bd65-eaa007d33239 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:34.130524 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:34.156794 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2697eb-75a4-4ecc-894e-ebb69e7f7018 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:34.238673 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf8f5d3-453f-466f-bd65-eaa007d33239 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:34.243821 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:34.275184 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2697eb-75a4-4ecc-894e-ebb69e7f7018 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:34.280356 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8f2dac-a490-49be-8006-7183ee4adde5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:34.281023 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f2dac-a490-49be-8006-7183ee4adde5 None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:34.281300 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f2dac-a490-49be-8006-7183ee4adde5 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:34.281570 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f2dac-a490-49be-8006-7183ee4adde5 None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:34.282247 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8f2dac-a490-49be-8006-7183ee4adde5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:34.283657 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8f2dac-a490-49be-8006-7183ee4adde5 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:34.286320 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: WARNING keystone.server.flask.common [None req-fe8f2dac-a490-49be-8006-7183ee4adde5 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 28 15:29:34.300621 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.sql.core [None req-fe8f2dac-a490-49be-8006-7183ee4adde5 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 28 15:29:34.300621 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 28 15:29:34.300621 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [parameters: {'user_id': 'f88d8098d5004b4fb25da1c08ea0a1a3', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 28 15:29:34.300621 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81604) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 28 15:29:34.321496 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e51beb-fba9-44af-a4c1-d1997696ab91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:34.322514 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e51beb-fba9-44af-a4c1-d1997696ab91 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:34.323003 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e51beb-fba9-44af-a4c1-d1997696ab91 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:34.323501 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e51beb-fba9-44af-a4c1-d1997696ab91 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:34.323954 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-fe8f2dac-a490-49be-8006-7183ee4adde5 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 28 15:29:34.323954 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:34.323954 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 28 15:29:34.323954 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 28 15:29:34.323954 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 28 15:29:34.323954 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 28 15:29:34.323954 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 28 15:29:34.323954 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application result = self._query(query) Sep 28 15:29:34.323954 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 28 15:29:34.323954 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application conn.query(q) Sep 28 15:29:34.323954 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 28 15:29:34.323954 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 28 15:29:34.323954 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 28 15:29:34.323954 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application result.read() Sep 28 15:29:34.323954 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 28 15:29:34.324813 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 28 15:29:34.324813 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 28 15:29:34.324813 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 28 15:29:34.324813 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 28 15:29:34.324813 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 28 15:29:34.324813 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 28 15:29:34.324813 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 28 15:29:34.324813 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 28 15:29:34.324813 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:34.324813 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 28 15:29:34.324813 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:34.324813 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:34.324813 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 28 15:29:34.324813 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 28 15:29:34.324813 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 28 15:29:34.324813 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 28 15:29:34.324813 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 28 15:29:34.324813 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application next(self.gen) Sep 28 15:29:34.325778 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 28 15:29:34.325778 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application yield resource Sep 28 15:29:34.325778 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 28 15:29:34.325778 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application next(self.gen) Sep 28 15:29:34.325778 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 28 15:29:34.325778 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self.session.rollback() Sep 28 15:29:34.325778 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 28 15:29:34.325778 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self.force_reraise() Sep 28 15:29:34.325778 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 28 15:29:34.325778 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise self.value Sep 28 15:29:34.325778 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 28 15:29:34.325778 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 28 15:29:34.325778 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 28 15:29:34.325778 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application session.commit() Sep 28 15:29:34.325778 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 28 15:29:34.325778 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 28 15:29:34.325778 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 28 15:29:34.326723 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._prepare_impl() Sep 28 15:29:34.326723 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 28 15:29:34.326723 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self.session.flush() Sep 28 15:29:34.326723 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 28 15:29:34.326723 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._flush(objects) Sep 28 15:29:34.326723 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 28 15:29:34.326723 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 28 15:29:34.326723 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 28 15:29:34.326723 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application compat.raise_( Sep 28 15:29:34.326723 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 28 15:29:34.326723 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception Sep 28 15:29:34.326723 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 28 15:29:34.326723 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application flush_context.execute() Sep 28 15:29:34.326723 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 28 15:29:34.326723 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rec.execute(self) Sep 28 15:29:34.326723 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 28 15:29:34.326723 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 28 15:29:34.327635 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 28 15:29:34.327635 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 28 15:29:34.327635 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 28 15:29:34.327635 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 28 15:29:34.327635 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 28 15:29:34.327635 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 28 15:29:34.327635 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 28 15:29:34.327635 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 28 15:29:34.327635 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 28 15:29:34.327635 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 28 15:29:34.327635 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 28 15:29:34.327635 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 28 15:29:34.327635 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 28 15:29:34.327635 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 28 15:29:34.327635 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 28 15:29:34.327635 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception Sep 28 15:29:34.328480 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 28 15:29:34.328480 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 28 15:29:34.328480 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 28 15:29:34.328480 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 28 15:29:34.328480 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 28 15:29:34.328480 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application result = self._query(query) Sep 28 15:29:34.328480 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 28 15:29:34.328480 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application conn.query(q) Sep 28 15:29:34.328480 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 28 15:29:34.328480 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 28 15:29:34.328480 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 28 15:29:34.328480 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application result.read() Sep 28 15:29:34.328480 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 28 15:29:34.328480 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 28 15:29:34.328480 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 28 15:29:34.328480 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 28 15:29:34.329314 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 28 15:29:34.329314 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 28 15:29:34.329314 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 28 15:29:34.329314 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 28 15:29:34.329314 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 28 15:29:34.329314 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 28 15:29:34.329314 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [parameters: {'user_id': 'f88d8098d5004b4fb25da1c08ea0a1a3', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 28 15:29:34.329314 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 28 15:29:34.329314 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:34.329314 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:29:34.329314 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:34.329314 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:34.329314 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:34.329314 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:34.329314 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:34.331718 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:34.331718 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:34.331718 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:34.331718 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:34.331718 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:34.331718 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:34.331718 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:34.331718 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 28 15:29:34.331718 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 28 15:29:34.331718 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:34.331718 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:34.331718 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:29:34.331718 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:34.331718 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:29:34.331718 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:34.331718 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 28 15:29:34.331718 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 28 15:29:34.334491 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 28 15:29:34.334491 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 28 15:29:34.334491 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 28 15:29:34.334491 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 28 15:29:34.334491 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 28 15:29:34.334491 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:34.334491 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 486/968] 10.222.0.52 () {60 vars in 1131 bytes} [Wed Sep 28 15:29:34 2022] POST /identity/v3/users => generated 160 bytes in 198 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:34.334491 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8f2dac-a490-49be-8006-7183ee4adde5 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:34.353974 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e51beb-fba9-44af-a4c1-d1997696ab91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:34.359028 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-d0e51beb-fba9-44af-a4c1-d1997696ab91 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 15:29:34.359028 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:34.359028 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:34.359028 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:34.359028 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:34.359028 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:34.359028 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:34.359028 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:34.359028 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:34.359028 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:34.359028 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:34.359028 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:34.359028 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:34.359028 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:34.359028 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:34.359028 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:34.360925 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:34.360925 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:34.360925 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:34.360925 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:34.360925 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:34.360925 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:34.360925 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:34.360925 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:34.360925 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:34.360925 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:34.360925 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:34.360925 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:34.360925 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:34.360925 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:34.360925 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:34.360925 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:34.361930 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:34.361930 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:34.361930 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:34.361930 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:34.361930 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:34.361930 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:34.361930 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:34.361930 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:34.361930 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:34.361930 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:34.361930 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 15:29:34.361930 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:34.361930 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 483/969] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Sep 28 15:29:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:34.374851 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e51beb-fba9-44af-a4c1-d1997696ab91 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:34.414433 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:34.464057 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8f2dac-a490-49be-8006-7183ee4adde5 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:34.482540 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e51beb-fba9-44af-a4c1-d1997696ab91 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:34.507734 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072d6c15-e5a4-481b-9e2e-eeb043976ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:34.508851 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d6c15-e5a4-481b-9e2e-eeb043976ebc None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:34.509272 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d6c15-e5a4-481b-9e2e-eeb043976ebc None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:34.509839 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d6c15-e5a4-481b-9e2e-eeb043976ebc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:34.520031 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072d6c15-e5a4-481b-9e2e-eeb043976ebc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:34.522207 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072d6c15-e5a4-481b-9e2e-eeb043976ebc None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:34.523701 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75625a21-29a5-43d6-ba06-8d6067c602c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:34.524645 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75625a21-29a5-43d6-ba06-8d6067c602c3 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:34.525553 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75625a21-29a5-43d6-ba06-8d6067c602c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:34.526058 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75625a21-29a5-43d6-ba06-8d6067c602c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:34.526909 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75625a21-29a5-43d6-ba06-8d6067c602c3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:34.527382 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75625a21-29a5-43d6-ba06-8d6067c602c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:34.528906 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75625a21-29a5-43d6-ba06-8d6067c602c3 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:34.535184 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:34.535824 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-072d6c15-e5a4-481b-9e2e-eeb043976ebc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 28 15:29:34.535824 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:34.535824 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:34.535824 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:34.535824 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:34.535824 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:34.535824 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:34.535824 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:34.535824 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:34.535824 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:34.535824 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:34.535824 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:34.535824 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:29:34.535824 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:29:34.535824 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:29:34.535824 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:29:34.536736 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:34.536736 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:34.536736 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:29:34.536736 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:34.536736 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:29:34.536736 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:34.536736 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:34.536736 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:34.536736 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:34.536736 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:34.536736 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:34.536736 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:34.536736 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:34.536736 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:34.536736 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:34.536736 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:34.536736 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:34.536736 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:34.542837 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:34.542837 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:34.542837 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:29:34.542837 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:29:34.542837 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:29:34.542837 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:29:34.542837 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:29:34.542837 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:29:34.542837 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 28 15:29:34.542837 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:34.542837 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 487/970] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:29:34 2022] GET /identity/v3/users/cinder => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:34.545398 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072d6c15-e5a4-481b-9e2e-eeb043976ebc None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:34.546957 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 484/971] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Sep 28 15:29:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:34.552968 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75625a21-29a5-43d6-ba06-8d6067c602c3 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:34.657563 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072d6c15-e5a4-481b-9e2e-eeb043976ebc None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:34.683539 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:34.683955 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75625a21-29a5-43d6-ba06-8d6067c602c3 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:34.696198 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae644e6-2229-411c-9aeb-0307be46a624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:34.697359 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae644e6-2229-411c-9aeb-0307be46a624 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:34.697359 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae644e6-2229-411c-9aeb-0307be46a624 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:34.697555 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae644e6-2229-411c-9aeb-0307be46a624 None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:34.698420 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae644e6-2229-411c-9aeb-0307be46a624 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:34.698791 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae644e6-2229-411c-9aeb-0307be46a624 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:34.700815 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae644e6-2229-411c-9aeb-0307be46a624 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:34.731924 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d12755a-3a21-43e9-a4d7-e6c5556d18d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:34.732912 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d12755a-3a21-43e9-a4d7-e6c5556d18d9 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:34.733364 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d12755a-3a21-43e9-a4d7-e6c5556d18d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:34.733803 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d12755a-3a21-43e9-a4d7-e6c5556d18d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:34.745247 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d12755a-3a21-43e9-a4d7-e6c5556d18d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:34.746803 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d12755a-3a21-43e9-a4d7-e6c5556d18d9 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:34.754305 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 488/972] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Sep 28 15:29:34 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:34.772279 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-5d12755a-3a21-43e9-a4d7-e6c5556d18d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:34.772279 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:34.772279 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:34.772279 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:34.772279 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:34.772279 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:34.772279 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:34.772279 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:34.772279 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:34.772279 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:34.772279 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:34.772279 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:34.772279 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:34.772279 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:34.772279 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:34.772279 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:34.773285 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:34.773285 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:34.773285 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:34.773285 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:34.773285 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:34.773285 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:34.773285 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:34.773285 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:34.773285 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:34.773285 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:34.773285 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:34.773285 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:34.773285 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:34.773285 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:34.773285 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:34.773285 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:34.773285 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:34.774224 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:34.774224 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:34.774224 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:34.774224 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 485/973] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:29:34.779719 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae644e6-2229-411c-9aeb-0307be46a624 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:34.819212 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a71b9e-57b8-44a6-b639-7600b1bcde5b None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:34.819481 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a71b9e-57b8-44a6-b639-7600b1bcde5b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:34.819777 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a71b9e-57b8-44a6-b639-7600b1bcde5b None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:34.821599 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 486/974] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:29:34.828575 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ef72f0-9c48-4e89-a824-845643bedc69 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:34.828869 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ef72f0-9c48-4e89-a824-845643bedc69 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:34.829129 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ef72f0-9c48-4e89-a824-845643bedc69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:34.850714 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-b7ef72f0-9c48-4e89-a824-845643bedc69 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:34.852233 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b7ef72f0-9c48-4e89-a824-845643bedc69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:34.903875 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae644e6-2229-411c-9aeb-0307be46a624 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:34.924470 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:34.951021 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609e2dc2-54e2-4c95-9ca2-0bb204102237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:34.951562 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609e2dc2-54e2-4c95-9ca2-0bb204102237 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:34.951868 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609e2dc2-54e2-4c95-9ca2-0bb204102237 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:34.952205 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609e2dc2-54e2-4c95-9ca2-0bb204102237 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:34.952872 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609e2dc2-54e2-4c95-9ca2-0bb204102237 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:34.953198 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609e2dc2-54e2-4c95-9ca2-0bb204102237 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:34.953351 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 487/975] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:34 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 15:29:34.959434 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609e2dc2-54e2-4c95-9ca2-0bb204102237 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:34.973402 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 489/976] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:34.980352 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ef72f0-9c48-4e89-a824-845643bedc69 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:35.083522 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ef72f0-9c48-4e89-a824-845643bedc69 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:35.120108 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b38b67b-e487-4f69-b323-bd1b968c20de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:35.120773 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b38b67b-e487-4f69-b323-bd1b968c20de None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:35.121051 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b38b67b-e487-4f69-b323-bd1b968c20de None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:35.121318 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b38b67b-e487-4f69-b323-bd1b968c20de None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:35.129659 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b38b67b-e487-4f69-b323-bd1b968c20de None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:35.132084 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b38b67b-e487-4f69-b323-bd1b968c20de None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:35.145081 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-3b38b67b-e487-4f69-b323-bd1b968c20de None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 15:29:35.145081 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:35.145081 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:35.145081 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:35.145081 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:35.145081 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:35.145081 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:35.145081 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:35.145081 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:35.145081 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:35.145081 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:35.145081 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:35.145081 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:29:35.145081 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:29:35.145081 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:29:35.145081 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:29:35.146952 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:35.146952 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:35.146952 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:29:35.146952 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:35.146952 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:29:35.146952 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:35.146952 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:35.146952 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:35.146952 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:35.146952 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:35.146952 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:35.146952 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:35.146952 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:35.146952 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:35.146952 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:35.146952 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:35.146952 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:35.146952 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:35.147934 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:35.147934 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:35.147934 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:29:35.147934 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:29:35.147934 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:29:35.147934 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:29:35.147934 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:29:35.147934 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:29:35.147934 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 28 15:29:35.147934 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:35.147934 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 488/977] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Sep 28 15:29:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:35.154786 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609e2dc2-54e2-4c95-9ca2-0bb204102237 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:35.204840 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:35.228002 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b38b67b-e487-4f69-b323-bd1b968c20de None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:35.264078 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609e2dc2-54e2-4c95-9ca2-0bb204102237 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:35.275000 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:35.301451 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f89d07-54d6-42fa-a860-62429d9068d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:35.302099 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f89d07-54d6-42fa-a860-62429d9068d2 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:35.302876 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f89d07-54d6-42fa-a860-62429d9068d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:35.303225 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f89d07-54d6-42fa-a860-62429d9068d2 None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:35.303943 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f89d07-54d6-42fa-a860-62429d9068d2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:35.304245 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f89d07-54d6-42fa-a860-62429d9068d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:35.305654 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f89d07-54d6-42fa-a860-62429d9068d2 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:35.322918 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 490/978] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Sep 28 15:29:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:35.324177 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a71b2-d59a-4431-9bbc-57b1216950ec None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:35.324460 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a71b2-d59a-4431-9bbc-57b1216950ec None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:35.324733 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a71b2-d59a-4431-9bbc-57b1216950ec None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:35.325964 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 491/979] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:35.331244 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a71b2-d59a-4431-9bbc-57b1216950ec None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:35.332520 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b38b67b-e487-4f69-b323-bd1b968c20de None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:35.365731 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f5c01a-424c-47cc-a594-d5d98ba14d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:35.366391 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5c01a-424c-47cc-a594-d5d98ba14d72 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:35.366667 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5c01a-424c-47cc-a594-d5d98ba14d72 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:35.366939 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5c01a-424c-47cc-a594-d5d98ba14d72 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:35.367508 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f5c01a-424c-47cc-a594-d5d98ba14d72 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:35.368863 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f5c01a-424c-47cc-a594-d5d98ba14d72 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:35.377724 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-75f5c01a-424c-47cc-a594-d5d98ba14d72 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 15:29:35.377724 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:35.377724 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:29:35.377724 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:29:35.377724 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:29:35.377724 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:29:35.377724 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:29:35.377724 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:29:35.377724 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 15:29:35.377724 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:35.377724 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:29:35.377724 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:35.377724 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:35.377724 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:35.377724 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:35.377724 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:35.377724 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:35.378759 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:35.378759 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:35.378759 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:35.378759 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:35.378759 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:35.378759 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:35.378759 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:29:35.378759 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:29:35.378759 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:29:35.378759 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:29:35.378759 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:35.378759 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:35.378759 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:35.378759 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:35.378759 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:35.378759 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:35.379700 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:35.379700 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:35.379700 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:35.379700 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:35.379700 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:35.379700 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:35.379700 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:35.379700 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:35.379700 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:35.379700 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:35.379700 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:29:35.379700 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:29:35.379700 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 15:29:35.379700 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:35.379700 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 489/980] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Sep 28 15:29:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:29:35.381334 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a4b0b-12d8-47ac-93f4-5958f1c97640 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:35.381334 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a4b0b-12d8-47ac-93f4-5958f1c97640 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:35.381334 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a4b0b-12d8-47ac-93f4-5958f1c97640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:35.398323 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-a39a4b0b-12d8-47ac-93f4-5958f1c97640 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:35.399492 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a39a4b0b-12d8-47ac-93f4-5958f1c97640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:35.430279 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:35.434686 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a71b2-d59a-4431-9bbc-57b1216950ec None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:35.463943 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 490/981] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:35 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:29:35.464688 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39a4b0b-12d8-47ac-93f4-5958f1c97640 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:35.467963 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0dce53-0d2d-4f4b-a51d-ae6d06825cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:35.468564 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0dce53-0d2d-4f4b-a51d-ae6d06825cd6 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:35.468840 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0dce53-0d2d-4f4b-a51d-ae6d06825cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:35.469109 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0dce53-0d2d-4f4b-a51d-ae6d06825cd6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:35.500469 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:35.544563 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0dce53-0d2d-4f4b-a51d-ae6d06825cd6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:35.546030 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0dce53-0d2d-4f4b-a51d-ae6d06825cd6 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:35.569822 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39a4b0b-12d8-47ac-93f4-5958f1c97640 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:35.606801 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65de828b-81cd-46eb-8d1e-a72ace084bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:35.607440 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de828b-81cd-46eb-8d1e-a72ace084bbc None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:35.607801 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de828b-81cd-46eb-8d1e-a72ace084bbc None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:35.608273 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de828b-81cd-46eb-8d1e-a72ace084bbc None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:35.608898 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65de828b-81cd-46eb-8d1e-a72ace084bbc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:35.609195 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65de828b-81cd-46eb-8d1e-a72ace084bbc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:35.611175 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65de828b-81cd-46eb-8d1e-a72ace084bbc None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:35.622630 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 491/982] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Sep 28 15:29:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:35.623673 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65de828b-81cd-46eb-8d1e-a72ace084bbc None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:35.649297 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:35.653439 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 492/983] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:29:35 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 15:29:35.658514 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0dce53-0d2d-4f4b-a51d-ae6d06825cd6 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:35.733001 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:35.736321 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65de828b-81cd-46eb-8d1e-a72ace084bbc None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:35.762924 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0dce53-0d2d-4f4b-a51d-ae6d06825cd6 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:35.770787 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fce78e9-49ed-4f0f-b3ff-ed03aa758ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:35.771962 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce78e9-49ed-4f0f-b3ff-ed03aa758ca5 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:35.772242 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce78e9-49ed-4f0f-b3ff-ed03aa758ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:35.772519 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce78e9-49ed-4f0f-b3ff-ed03aa758ca5 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:35.773131 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fce78e9-49ed-4f0f-b3ff-ed03aa758ca5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:35.773431 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fce78e9-49ed-4f0f-b3ff-ed03aa758ca5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:35.774827 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fce78e9-49ed-4f0f-b3ff-ed03aa758ca5 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:35.779614 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 492/984] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Sep 28 15:29:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:35.780211 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fce78e9-49ed-4f0f-b3ff-ed03aa758ca5 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:35.799503 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e379d97a-cb15-41d3-b3be-1ffdac7b137d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:35.800132 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e379d97a-cb15-41d3-b3be-1ffdac7b137d None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:35.800399 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e379d97a-cb15-41d3-b3be-1ffdac7b137d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:35.800662 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e379d97a-cb15-41d3-b3be-1ffdac7b137d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:35.810603 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e379d97a-cb15-41d3-b3be-1ffdac7b137d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:35.812032 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e379d97a-cb15-41d3-b3be-1ffdac7b137d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:35.826787 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-e379d97a-cb15-41d3-b3be-1ffdac7b137d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:35.826787 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:35.826787 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:35.826787 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:35.826787 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:35.826787 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:35.826787 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:35.826787 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:35.826787 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:35.826787 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:35.826787 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:35.826787 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:35.826787 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:35.826787 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:35.826787 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:35.826787 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:35.827789 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:35.827789 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:35.827789 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:35.827789 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:35.827789 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:35.827789 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:35.827789 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:35.827789 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:35.827789 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:35.827789 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:35.827789 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:35.827789 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:35.827789 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:35.827789 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:35.827789 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:35.827789 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:35.827789 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:35.829324 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:35.829324 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:35.829324 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:35.829324 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 493/985] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:29:35.829324 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e379d97a-cb15-41d3-b3be-1ffdac7b137d None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:35.878905 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fce78e9-49ed-4f0f-b3ff-ed03aa758ca5 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:35.879455 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:35.913758 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286f0927-0ff4-4099-a1a0-49c9815fc296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:35.914409 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286f0927-0ff4-4099-a1a0-49c9815fc296 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:35.914689 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286f0927-0ff4-4099-a1a0-49c9815fc296 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:35.914959 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286f0927-0ff4-4099-a1a0-49c9815fc296 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:35.915528 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286f0927-0ff4-4099-a1a0-49c9815fc296 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:35.916888 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286f0927-0ff4-4099-a1a0-49c9815fc296 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:35.925414 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-286f0927-0ff4-4099-a1a0-49c9815fc296 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 28 15:29:35.925414 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:35.925414 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:29:35.925414 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:29:35.925414 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:29:35.925414 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:29:35.925414 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:29:35.925414 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:29:35.925414 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 28 15:29:35.925414 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:35.925414 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:29:35.925414 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:35.925414 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:35.925414 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:35.925414 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:35.925414 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:35.925414 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:35.926634 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:35.926634 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:35.926634 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:35.926634 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:35.926634 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:35.926634 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:35.926634 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:29:35.926634 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:29:35.926634 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:29:35.926634 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:29:35.926634 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:35.926634 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:35.926634 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:35.926634 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:35.926634 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:35.926634 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:35.927546 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:35.927546 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:35.927546 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:35.927546 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:35.927546 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:35.927546 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:35.927546 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:35.927546 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:35.927546 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:35.927546 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:35.927546 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:29:35.927546 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:29:35.927546 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 28 15:29:35.927546 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:35.927546 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 493/986] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:29:35 2022] GET /identity/v3/services/compute => generated 88 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:35.928424 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e379d97a-cb15-41d3-b3be-1ffdac7b137d None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:35.928499 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286f0927-0ff4-4099-a1a0-49c9815fc296 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:35.956964 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:35.961366 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2790f-32f4-4486-a361-8c445536b0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:35.962002 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2790f-32f4-4486-a361-8c445536b0d8 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:35.962279 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2790f-32f4-4486-a361-8c445536b0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:35.962548 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2790f-32f4-4486-a361-8c445536b0d8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:35.963245 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc2790f-32f4-4486-a361-8c445536b0d8 None admin] RBAC: Adding query filter params (interface=admin, service_id=a1c25e40b06c4486a573c015c523c4b8, region_id=RegionOne) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:35.963551 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc2790f-32f4-4486-a361-8c445536b0d8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:35.964907 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc2790f-32f4-4486-a361-8c445536b0d8 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:35.969923 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 494/987] 10.222.0.52 () {56 vars in 1249 bytes} [Wed Sep 28 15:29:35 2022] GET /identity/v3/endpoints?service_id=a1c25e40b06c4486a573c015c523c4b8&interface=admin®ion_id=RegionOne => generated 193 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:35.970761 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2790f-32f4-4486-a361-8c445536b0d8 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:36.028456 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286f0927-0ff4-4099-a1a0-49c9815fc296 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:36.061027 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68383087-6369-4c89-87c8-187a3444ff2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:36.061621 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68383087-6369-4c89-87c8-187a3444ff2a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:36.061929 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68383087-6369-4c89-87c8-187a3444ff2a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:36.062217 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68383087-6369-4c89-87c8-187a3444ff2a None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:36.062841 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68383087-6369-4c89-87c8-187a3444ff2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:36.063137 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68383087-6369-4c89-87c8-187a3444ff2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:36.064542 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68383087-6369-4c89-87c8-187a3444ff2a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:36.071402 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2790f-32f4-4486-a361-8c445536b0d8 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:36.076729 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 494/988] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:36.077516 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68383087-6369-4c89-87c8-187a3444ff2a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:36.088222 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:36.105365 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab47056a-0d87-45d8-a56f-afcd2dcfee1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:36.106068 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab47056a-0d87-45d8-a56f-afcd2dcfee1b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:36.106408 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab47056a-0d87-45d8-a56f-afcd2dcfee1b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:36.106743 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab47056a-0d87-45d8-a56f-afcd2dcfee1b None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:36.107360 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab47056a-0d87-45d8-a56f-afcd2dcfee1b None admin] RBAC: Adding query filter params (name=compute) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:36.107659 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab47056a-0d87-45d8-a56f-afcd2dcfee1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:36.108996 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab47056a-0d87-45d8-a56f-afcd2dcfee1b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:36.113572 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 495/989] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Sep 28 15:29:35 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:36.114352 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab47056a-0d87-45d8-a56f-afcd2dcfee1b None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:36.165895 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:36.178737 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68383087-6369-4c89-87c8-187a3444ff2a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:36.212236 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf05445-f66b-4cc1-8634-a64919a57da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:36.212833 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf05445-f66b-4cc1-8634-a64919a57da7 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:36.213108 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf05445-f66b-4cc1-8634-a64919a57da7 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:36.213376 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf05445-f66b-4cc1-8634-a64919a57da7 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:36.214003 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf05445-f66b-4cc1-8634-a64919a57da7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:36.215333 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab47056a-0d87-45d8-a56f-afcd2dcfee1b None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:36.215431 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf05445-f66b-4cc1-8634-a64919a57da7 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:36.221329 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 495/990] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Sep 28 15:29:36 2022] GET /identity/v3/services => generated 1727 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:29:36.222120 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf05445-f66b-4cc1-8634-a64919a57da7 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:36.260020 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca439aa-72b1-4175-b4c8-c4bc40652010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:36.260664 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca439aa-72b1-4175-b4c8-c4bc40652010 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:36.260936 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca439aa-72b1-4175-b4c8-c4bc40652010 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:36.261203 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca439aa-72b1-4175-b4c8-c4bc40652010 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:36.268625 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca439aa-72b1-4175-b4c8-c4bc40652010 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:36.270108 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca439aa-72b1-4175-b4c8-c4bc40652010 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:36.287489 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-0ca439aa-72b1-4175-b4c8-c4bc40652010 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:36.287489 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:36.287489 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:36.287489 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:36.287489 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:36.287489 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:36.287489 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:36.287489 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:36.287489 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:36.287489 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:36.287489 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:36.287489 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:36.287489 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:29:36.287489 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:29:36.287489 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:29:36.288414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:29:36.288414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:36.288414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:36.288414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:36.288414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:36.288414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:36.288414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:36.288414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:36.288414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:36.288414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:36.288414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:36.288414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:36.288414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:36.288414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:36.288414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:36.288414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:36.288414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:36.289321 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:29:36.289321 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:29:36.289321 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:29:36.289321 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:29:36.289321 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:29:36.289321 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:29:36.289321 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:36.289321 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:36.289321 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 496/991] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:29:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:36.294520 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca439aa-72b1-4175-b4c8-c4bc40652010 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:36.343447 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf05445-f66b-4cc1-8634-a64919a57da7 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:36.350914 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:36.383174 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc45953-f1ce-404f-aafc-b0c0059fedaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:36.383892 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc45953-f1ce-404f-aafc-b0c0059fedaa None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:36.384173 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc45953-f1ce-404f-aafc-b0c0059fedaa None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:36.384440 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc45953-f1ce-404f-aafc-b0c0059fedaa None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:36.385108 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc45953-f1ce-404f-aafc-b0c0059fedaa None admin] RBAC: Adding query filter params (type=compute) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:36.385354 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc45953-f1ce-404f-aafc-b0c0059fedaa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:36.386756 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc45953-f1ce-404f-aafc-b0c0059fedaa None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:36.392410 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 496/992] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Sep 28 15:29:36 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:36.398117 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc45953-f1ce-404f-aafc-b0c0059fedaa None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:36.417480 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:36.422388 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca439aa-72b1-4175-b4c8-c4bc40652010 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:36.456975 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef54e5fa-0d21-4ee9-a351-d44f5d528e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:36.457579 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef54e5fa-0d21-4ee9-a351-d44f5d528e11 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:36.457880 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef54e5fa-0d21-4ee9-a351-d44f5d528e11 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:36.458152 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef54e5fa-0d21-4ee9-a351-d44f5d528e11 None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:36.459382 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef54e5fa-0d21-4ee9-a351-d44f5d528e11 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:36.459690 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef54e5fa-0d21-4ee9-a351-d44f5d528e11 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:36.461113 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef54e5fa-0d21-4ee9-a351-d44f5d528e11 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:36.474483 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 497/993] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 15:29:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:36.480359 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef54e5fa-0d21-4ee9-a351-d44f5d528e11 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:36.503011 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc45953-f1ce-404f-aafc-b0c0059fedaa None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:36.539913 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82ce96b-c23c-4369-a033-32d71c2c041e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:36.540774 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ce96b-c23c-4369-a033-32d71c2c041e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:36.541061 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ce96b-c23c-4369-a033-32d71c2c041e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:36.541335 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ce96b-c23c-4369-a033-32d71c2c041e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:36.542053 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82ce96b-c23c-4369-a033-32d71c2c041e None admin] RBAC: Adding query filter params (interface=public, service_id=cbac371e82084a1a848481351be5c3e5, region_id=RegionOne) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:36.542362 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82ce96b-c23c-4369-a033-32d71c2c041e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:36.543729 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82ce96b-c23c-4369-a033-32d71c2c041e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:36.549313 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 497/994] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Sep 28 15:29:36 2022] GET /identity/v3/endpoints?service_id=cbac371e82084a1a848481351be5c3e5&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:36.554807 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82ce96b-c23c-4369-a033-32d71c2c041e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:36.587309 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:36.590511 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef54e5fa-0d21-4ee9-a351-d44f5d528e11 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:36.626224 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331a1068-7e04-4f31-9cdb-7b7b638449cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:36.626830 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a1068-7e04-4f31-9cdb-7b7b638449cb None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:36.627101 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a1068-7e04-4f31-9cdb-7b7b638449cb None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:36.627366 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a1068-7e04-4f31-9cdb-7b7b638449cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:36.628077 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331a1068-7e04-4f31-9cdb-7b7b638449cb None admin] RBAC: Adding query filter params (role.id=769ffe2b49154917b28e916692c53d5f, scope.project.id=3dc6c5f5fcd34922b0d09cd2aaed6cf1, user.id=c81e5fb007614cf2992f5e02e85d6307) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:36.628378 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331a1068-7e04-4f31-9cdb-7b7b638449cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:36.630164 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331a1068-7e04-4f31-9cdb-7b7b638449cb None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:36.637447 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 498/995] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 15:29:36 2022] GET /identity/v3/role_assignments?user.id=c81e5fb007614cf2992f5e02e85d6307&scope.project.id=3dc6c5f5fcd34922b0d09cd2aaed6cf1&role.id=769ffe2b49154917b28e916692c53d5f => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:36.645328 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:36.667998 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82ce96b-c23c-4369-a033-32d71c2c041e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:36.705864 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c028dc0b-1651-4109-91af-b67e1b5d7b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:36.706515 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c028dc0b-1651-4109-91af-b67e1b5d7b2c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:36.706792 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c028dc0b-1651-4109-91af-b67e1b5d7b2c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:36.707058 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c028dc0b-1651-4109-91af-b67e1b5d7b2c None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:36.707688 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c028dc0b-1651-4109-91af-b67e1b5d7b2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:36.711320 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c028dc0b-1651-4109-91af-b67e1b5d7b2c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:36.721975 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 498/996] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Sep 28 15:29:36 2022] GET /identity/v3/services => generated 1727 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:29:38.409993 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb3a98-8d00-4ff4-8aad-48cbbf0be95c None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:38.410530 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb3a98-8d00-4ff4-8aad-48cbbf0be95c None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:38.410530 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb3a98-8d00-4ff4-8aad-48cbbf0be95c None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:38.412113 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 499/997] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:38.417297 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc10ee-4170-419f-a882-c47385cf254a None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:38.417788 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc10ee-4170-419f-a882-c47385cf254a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:38.417993 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc10ee-4170-419f-a882-c47385cf254a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:38.436356 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d6cc10ee-4170-419f-a882-c47385cf254a None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:38.437527 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d6cc10ee-4170-419f-a882-c47385cf254a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:38.504269 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 499/998] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:38 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:29:38.773417 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb3a98-8d00-4ff4-8aad-48cbbf0be95c None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:38.831596 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c78d5e-3a11-4e30-b67b-c230e8fe30a7 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:38.832286 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c78d5e-3a11-4e30-b67b-c230e8fe30a7 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:38.832773 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c78d5e-3a11-4e30-b67b-c230e8fe30a7 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:38.834743 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 500/999] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:29:38.839556 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51de9f5-040f-42f3-bc45-7e97e033cfb6 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:38.839839 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51de9f5-040f-42f3-bc45-7e97e033cfb6 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:38.840108 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51de9f5-040f-42f3-bc45-7e97e033cfb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:38.858116 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-f51de9f5-040f-42f3-bc45-7e97e033cfb6 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:38.859299 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f51de9f5-040f-42f3-bc45-7e97e033cfb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:38.892359 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb3a98-8d00-4ff4-8aad-48cbbf0be95c None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:38.926183 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d5c6b-5345-40a7-b55d-066462e6832b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:38.926805 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d5c6b-5345-40a7-b55d-066462e6832b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:38.927081 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d5c6b-5345-40a7-b55d-066462e6832b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:38.927904 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5d5c6b-5345-40a7-b55d-066462e6832b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:38.928502 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d5c6b-5345-40a7-b55d-066462e6832b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:38.929887 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5d5c6b-5345-40a7-b55d-066462e6832b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:38.931435 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 501/1000] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:38 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 15:29:38.932824 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fafae5-41bb-4eae-8bb1-aa1c78a487d5 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:38.933117 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fafae5-41bb-4eae-8bb1-aa1c78a487d5 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:38.933392 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fafae5-41bb-4eae-8bb1-aa1c78a487d5 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:38.934812 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 502/1001] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:38.941357 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-eb5d5c6b-5345-40a7-b55d-066462e6832b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 15:29:38.941357 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:38.941357 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:29:38.941357 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:29:38.941357 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:29:38.941357 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:29:38.941357 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:29:38.941357 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:29:38.941357 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 15:29:38.941357 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:38.941357 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:29:38.941357 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:38.941357 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:38.941357 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:38.941357 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:38.941357 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:38.941357 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:38.942404 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:38.942404 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:38.942404 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:38.942404 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:38.942404 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:38.942404 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:38.942404 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:29:38.942404 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:29:38.942404 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:29:38.942404 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:29:38.942404 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:38.942404 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:38.942404 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:38.942404 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:38.942404 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:38.942404 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:38.947059 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:38.947059 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:38.947059 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:38.947059 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:38.947059 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:38.947059 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:38.947059 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:38.947059 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:38.947059 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:38.947059 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:38.947059 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:29:38.947059 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:29:38.947059 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 28 15:29:38.947059 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:38.947059 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 500/1002] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Sep 28 15:29:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:38.948021 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a548f86-bb8d-4aca-a7ce-ab8b8eb95ccc None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:38.948021 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a548f86-bb8d-4aca-a7ce-ab8b8eb95ccc None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:38.948021 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a548f86-bb8d-4aca-a7ce-ab8b8eb95ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:38.948527 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d5c6b-5345-40a7-b55d-066462e6832b None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:38.953695 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:38.966006 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-7a548f86-bb8d-4aca-a7ce-ab8b8eb95ccc None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:38.967253 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7a548f86-bb8d-4aca-a7ce-ab8b8eb95ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:38.981056 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:39.044552 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 503/1003] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:38 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:29:39.057959 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5d5c6b-5345-40a7-b55d-066462e6832b None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:39.092101 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2929c315-a246-4fca-9448-9ef3f3487fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:39.092816 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2929c315-a246-4fca-9448-9ef3f3487fc3 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:39.093157 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2929c315-a246-4fca-9448-9ef3f3487fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:39.093487 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2929c315-a246-4fca-9448-9ef3f3487fc3 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:39.094230 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2929c315-a246-4fca-9448-9ef3f3487fc3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:39.094588 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2929c315-a246-4fca-9448-9ef3f3487fc3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:39.096014 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2929c315-a246-4fca-9448-9ef3f3487fc3 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:39.102003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 501/1004] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Sep 28 15:29:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:39.108322 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a548f86-bb8d-4aca-a7ce-ab8b8eb95ccc None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:39.198076 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2929c315-a246-4fca-9448-9ef3f3487fc3 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:39.209339 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a548f86-bb8d-4aca-a7ce-ab8b8eb95ccc None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:39.245945 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ee223-1899-4161-a471-31e013c73665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:39.246755 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ee223-1899-4161-a471-31e013c73665 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:39.247195 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ee223-1899-4161-a471-31e013c73665 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:39.247627 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ee223-1899-4161-a471-31e013c73665 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:39.248382 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3ee223-1899-4161-a471-31e013c73665 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:39.248819 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3ee223-1899-4161-a471-31e013c73665 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:39.250341 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3ee223-1899-4161-a471-31e013c73665 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:39.252671 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:39.261730 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 504/1005] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Sep 28 15:29:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:39.263099 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ee223-1899-4161-a471-31e013c73665 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:39.297563 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:39.307410 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2929c315-a246-4fca-9448-9ef3f3487fc3 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:39.346004 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7120d9-d0e5-4e00-b7d5-7366dffe4a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:39.346750 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7120d9-d0e5-4e00-b7d5-7366dffe4a76 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:39.347028 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7120d9-d0e5-4e00-b7d5-7366dffe4a76 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:39.347301 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7120d9-d0e5-4e00-b7d5-7366dffe4a76 None admin] PATH_INFO: `/v3/services/image` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:39.347874 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7120d9-d0e5-4e00-b7d5-7366dffe4a76 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:39.349337 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7120d9-d0e5-4e00-b7d5-7366dffe4a76 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:39.357911 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-4e7120d9-d0e5-4e00-b7d5-7366dffe4a76 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 28 15:29:39.357911 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:39.357911 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:29:39.357911 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:29:39.357911 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:29:39.357911 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:29:39.357911 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:29:39.357911 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:29:39.357911 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 28 15:29:39.357911 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:39.357911 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:29:39.357911 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:39.357911 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:39.357911 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:39.357911 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:39.357911 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:39.357911 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:39.358892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:39.358892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:39.358892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:39.358892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:39.358892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:39.358892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:39.358892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:29:39.358892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:29:39.358892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:29:39.358892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:29:39.358892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:39.358892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:39.358892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:39.358892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:39.358892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:39.358892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:39.359800 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:39.359800 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:39.359800 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:39.359800 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:39.359800 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:39.359800 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:39.359800 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:39.359800 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:39.359800 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:39.359800 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:39.359800 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:29:39.359800 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:29:39.359800 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 28 15:29:39.359800 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:39.359800 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 502/1006] 10.222.0.52 () {56 vars in 1105 bytes} [Wed Sep 28 15:29:39 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:39.360637 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7120d9-d0e5-4e00-b7d5-7366dffe4a76 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:39.381746 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ee223-1899-4161-a471-31e013c73665 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:39.414354 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5943910b-d54f-4fe7-968a-aa12274f25d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:39.414949 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5943910b-d54f-4fe7-968a-aa12274f25d8 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:39.415227 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5943910b-d54f-4fe7-968a-aa12274f25d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:39.415503 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5943910b-d54f-4fe7-968a-aa12274f25d8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:39.416150 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5943910b-d54f-4fe7-968a-aa12274f25d8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:39.417495 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5943910b-d54f-4fe7-968a-aa12274f25d8 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:39.434808 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 505/1007] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:29:39 2022] POST /identity/v3/endpoints => generated 325 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:29:39.435874 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5943910b-d54f-4fe7-968a-aa12274f25d8 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:39.464394 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7120d9-d0e5-4e00-b7d5-7366dffe4a76 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:39.512322 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:39.513588 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204bba39-1bdf-4f3b-ad0a-22ebfc02ef67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:39.514430 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204bba39-1bdf-4f3b-ad0a-22ebfc02ef67 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:39.514599 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204bba39-1bdf-4f3b-ad0a-22ebfc02ef67 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:39.514950 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204bba39-1bdf-4f3b-ad0a-22ebfc02ef67 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:39.515350 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204bba39-1bdf-4f3b-ad0a-22ebfc02ef67 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:39.516778 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204bba39-1bdf-4f3b-ad0a-22ebfc02ef67 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:39.528441 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-204bba39-1bdf-4f3b-ad0a-22ebfc02ef67 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 28 15:29:39.528441 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:39.528441 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:29:39.528441 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:29:39.528441 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:29:39.528441 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:29:39.528441 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:29:39.528441 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:29:39.528441 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 28 15:29:39.528441 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:39.528441 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:29:39.528441 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:39.528441 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:39.528441 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:39.528441 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:39.528441 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:39.528441 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:39.529573 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:39.529573 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:39.529573 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:39.529573 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:39.529573 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:39.529573 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:39.529573 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:29:39.529573 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:29:39.529573 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:29:39.529573 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:29:39.529573 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:39.529573 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:39.529573 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:39.529573 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:39.529573 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:39.529573 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:39.530513 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:39.530513 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:39.530513 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:39.530513 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:39.530513 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:39.530513 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:39.530513 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:39.530513 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:39.530513 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:39.530513 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:39.530513 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:29:39.530513 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:29:39.530513 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 28 15:29:39.530513 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:39.530513 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 503/1008] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:29:39 2022] GET /identity/v3/services/compute => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:39.536250 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204bba39-1bdf-4f3b-ad0a-22ebfc02ef67 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:39.557639 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:39.565407 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5943910b-d54f-4fe7-968a-aa12274f25d8 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:39.608149 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf9492d-a92e-44d7-b0a4-c23aca782111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:39.608760 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf9492d-a92e-44d7-b0a4-c23aca782111 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:39.609447 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf9492d-a92e-44d7-b0a4-c23aca782111 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:39.609447 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf9492d-a92e-44d7-b0a4-c23aca782111 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:39.609933 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf9492d-a92e-44d7-b0a4-c23aca782111 None admin] RBAC: Adding query filter params (name=image) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:39.610233 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf9492d-a92e-44d7-b0a4-c23aca782111 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:39.612106 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf9492d-a92e-44d7-b0a4-c23aca782111 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:39.617951 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 506/1009] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:39 2022] GET /identity/v3/services?name=image => generated 122 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:39.623495 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf9492d-a92e-44d7-b0a4-c23aca782111 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:39.655027 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204bba39-1bdf-4f3b-ad0a-22ebfc02ef67 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:39.690720 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac9c685-e1c5-4e74-ba69-df3b31fdd34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:39.691324 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac9c685-e1c5-4e74-ba69-df3b31fdd34f None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:39.691598 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac9c685-e1c5-4e74-ba69-df3b31fdd34f None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:39.691866 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac9c685-e1c5-4e74-ba69-df3b31fdd34f None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:39.692487 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac9c685-e1c5-4e74-ba69-df3b31fdd34f None admin] RBAC: Adding query filter params (name=compute) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:39.693428 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac9c685-e1c5-4e74-ba69-df3b31fdd34f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:39.694825 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac9c685-e1c5-4e74-ba69-df3b31fdd34f None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:39.699555 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 504/1010] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Sep 28 15:29:39 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:39.704987 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac9c685-e1c5-4e74-ba69-df3b31fdd34f None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:39.741677 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf9492d-a92e-44d7-b0a4-c23aca782111 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:39.766333 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:39.779868 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a285a25-e668-4944-a712-b5881c70c504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:39.780480 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a285a25-e668-4944-a712-b5881c70c504 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:39.780765 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a285a25-e668-4944-a712-b5881c70c504 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:39.781040 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a285a25-e668-4944-a712-b5881c70c504 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:39.781667 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a285a25-e668-4944-a712-b5881c70c504 None admin] RBAC: Adding query filter params (type=image) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:39.782000 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a285a25-e668-4944-a712-b5881c70c504 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:39.783372 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a285a25-e668-4944-a712-b5881c70c504 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:39.788178 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 507/1011] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:39 2022] GET /identity/v3/services?type=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:39.790659 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:39.813533 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac9c685-e1c5-4e74-ba69-df3b31fdd34f None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:39.851494 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb96c3e2-2a9e-4fd4-801b-2c27895e38aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:39.852132 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb96c3e2-2a9e-4fd4-801b-2c27895e38aa None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:39.852409 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb96c3e2-2a9e-4fd4-801b-2c27895e38aa None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:39.852676 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb96c3e2-2a9e-4fd4-801b-2c27895e38aa None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:39.853314 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb96c3e2-2a9e-4fd4-801b-2c27895e38aa None admin] RBAC: Adding query filter params (type=compute) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:39.853614 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb96c3e2-2a9e-4fd4-801b-2c27895e38aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:39.855031 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb96c3e2-2a9e-4fd4-801b-2c27895e38aa None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:39.865871 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 505/1012] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Sep 28 15:29:39 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:39.867066 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a285a25-e668-4944-a712-b5881c70c504 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:39.979587 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a285a25-e668-4944-a712-b5881c70c504 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:40.015243 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef40c498-ca66-4037-acb6-e56f3dd2aa4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:40.015853 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef40c498-ca66-4037-acb6-e56f3dd2aa4b None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:40.016132 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef40c498-ca66-4037-acb6-e56f3dd2aa4b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:40.016401 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef40c498-ca66-4037-acb6-e56f3dd2aa4b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:40.017066 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef40c498-ca66-4037-acb6-e56f3dd2aa4b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:40.018462 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef40c498-ca66-4037-acb6-e56f3dd2aa4b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:40.045907 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 508/1013] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:29:39 2022] POST /identity/v3/endpoints => generated 334 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:29:41.600984 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a441ca3-68e7-4b10-9d5a-f2f7af6cca59 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:41.601502 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a441ca3-68e7-4b10-9d5a-f2f7af6cca59 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:41.601502 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a441ca3-68e7-4b10-9d5a-f2f7af6cca59 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:41.602900 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 506/1014] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:41.608139 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86d6d4-92cd-42d4-9ddf-fdbf412da632 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:41.608513 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86d6d4-92cd-42d4-9ddf-fdbf412da632 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:41.608690 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86d6d4-92cd-42d4-9ddf-fdbf412da632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:41.627630 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-0e86d6d4-92cd-42d4-9ddf-fdbf412da632 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:41.628787 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0e86d6d4-92cd-42d4-9ddf-fdbf412da632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:41.665170 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:41.704401 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 509/1015] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:41 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:29:41.953944 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125cabb5-b95f-4cfc-bf50-36e43c82c834 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:41.954252 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125cabb5-b95f-4cfc-bf50-36e43c82c834 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:41.954433 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125cabb5-b95f-4cfc-bf50-36e43c82c834 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:41.955865 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 507/1016] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:41.961570 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029949f-927b-4c20-882c-eaf3de6bb6a4 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:41.961950 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029949f-927b-4c20-882c-eaf3de6bb6a4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:41.962289 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029949f-927b-4c20-882c-eaf3de6bb6a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:41.968781 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125cabb5-b95f-4cfc-bf50-36e43c82c834 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:41.983446 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d029949f-927b-4c20-882c-eaf3de6bb6a4 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:41.984631 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d029949f-927b-4c20-882c-eaf3de6bb6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:42.057957 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 510/1017] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:41 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:29:42.078530 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125cabb5-b95f-4cfc-bf50-36e43c82c834 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:42.113524 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cc4552-8c5a-4eb9-acf8-f5d60e738d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:42.114163 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cc4552-8c5a-4eb9-acf8-f5d60e738d12 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:42.114539 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cc4552-8c5a-4eb9-acf8-f5d60e738d12 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:42.114783 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cc4552-8c5a-4eb9-acf8-f5d60e738d12 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:42.115861 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cc4552-8c5a-4eb9-acf8-f5d60e738d12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:42.117276 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cc4552-8c5a-4eb9-acf8-f5d60e738d12 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:42.119083 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 508/1018] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:42.125029 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d029949f-927b-4c20-882c-eaf3de6bb6a4 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:42.220443 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:42.225191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d029949f-927b-4c20-882c-eaf3de6bb6a4 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:42.264645 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0936dc-7353-4791-8a0f-674fb7452f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:42.265267 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0936dc-7353-4791-8a0f-674fb7452f6d None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:42.265541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0936dc-7353-4791-8a0f-674fb7452f6d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:42.265808 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0936dc-7353-4791-8a0f-674fb7452f6d None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:42.266485 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0936dc-7353-4791-8a0f-674fb7452f6d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:42.267921 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0936dc-7353-4791-8a0f-674fb7452f6d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:42.303345 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 511/1019] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Sep 28 15:29:42 2022] POST /identity/v3/projects => generated 311 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:29:42.329801 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cc4552-8c5a-4eb9-acf8-f5d60e738d12 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:42.347559 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:42.360244 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec6843-19b0-4f32-aa76-26c1ac68e155 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:42.360554 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec6843-19b0-4f32-aa76-26c1ac68e155 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:42.360816 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec6843-19b0-4f32-aa76-26c1ac68e155 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:42.362241 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 512/1020] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:29:42.368704 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc983c-de16-4782-9c76-f881a7525728 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:42.369101 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc983c-de16-4782-9c76-f881a7525728 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:42.369324 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc983c-de16-4782-9c76-f881a7525728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:42.390489 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-67fc983c-de16-4782-9c76-f881a7525728 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:42.391720 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-67fc983c-de16-4782-9c76-f881a7525728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:42.449440 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cc4552-8c5a-4eb9-acf8-f5d60e738d12 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:42.478826 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 513/1021] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:42 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 15:29:42.486340 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329dbd78-307e-4996-b262-9033d1e23382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:42.486946 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329dbd78-307e-4996-b262-9033d1e23382 None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:42.487226 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329dbd78-307e-4996-b262-9033d1e23382 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:42.487495 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329dbd78-307e-4996-b262-9033d1e23382 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:42.488156 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329dbd78-307e-4996-b262-9033d1e23382 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:42.489516 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329dbd78-307e-4996-b262-9033d1e23382 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:42.506784 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 509/1022] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Sep 28 15:29:42 2022] POST /identity/v3/services => generated 242 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:29:42.756282 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fc983c-de16-4782-9c76-f881a7525728 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:42.834486 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:42.857159 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fc983c-de16-4782-9c76-f881a7525728 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:42.897620 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcbcb91-e2d6-4428-ab5f-a67c6f81807d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:42.898281 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcbcb91-e2d6-4428-ab5f-a67c6f81807d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:42.898557 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcbcb91-e2d6-4428-ab5f-a67c6f81807d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:42.898825 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcbcb91-e2d6-4428-ab5f-a67c6f81807d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:42.908831 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcbcb91-e2d6-4428-ab5f-a67c6f81807d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:42.914217 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-edcbcb91-e2d6-4428-ab5f-a67c6f81807d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 15:29:42.914217 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:42.914217 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:42.914217 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:42.914217 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:42.914217 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:42.914217 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:42.914217 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:42.914217 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:42.914217 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:42.914217 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:42.914217 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:42.914217 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:42.914217 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:42.914217 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:42.914217 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:42.915233 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:42.915233 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:42.915233 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:42.915233 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:42.915233 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:42.915233 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:42.915233 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:42.915233 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:42.915233 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:42.915233 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:42.915233 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:42.915233 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:42.915233 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:42.915233 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:42.915233 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:42.915233 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:42.916671 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:42.916671 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:42.916671 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:42.916671 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:42.916671 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:42.916671 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:42.916671 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:42.916671 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:42.916671 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:42.916671 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:42.916671 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 15:29:42.916671 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:42.917545 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 514/1023] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Sep 28 15:29:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:42.921391 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329dbd78-307e-4996-b262-9033d1e23382 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:42.992929 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:43.029943 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329dbd78-307e-4996-b262-9033d1e23382 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:43.068466 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d3db7a-2515-4ab3-af80-8290668a821a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:43.069092 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d3db7a-2515-4ab3-af80-8290668a821a None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:43.069367 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d3db7a-2515-4ab3-af80-8290668a821a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:43.069633 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d3db7a-2515-4ab3-af80-8290668a821a None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:43.070318 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d3db7a-2515-4ab3-af80-8290668a821a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:43.070618 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d3db7a-2515-4ab3-af80-8290668a821a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:43.071965 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d3db7a-2515-4ab3-af80-8290668a821a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:43.086488 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 510/1024] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Sep 28 15:29:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:43.088299 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcbcb91-e2d6-4428-ab5f-a67c6f81807d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:43.187096 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcbcb91-e2d6-4428-ab5f-a67c6f81807d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:43.221325 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db66c2ef-40d4-4601-b835-5007abc591c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:43.221971 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66c2ef-40d4-4601-b835-5007abc591c4 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:43.222248 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66c2ef-40d4-4601-b835-5007abc591c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:43.222515 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66c2ef-40d4-4601-b835-5007abc591c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:43.225159 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:43.234280 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db66c2ef-40d4-4601-b835-5007abc591c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:43.235732 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db66c2ef-40d4-4601-b835-5007abc591c4 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:43.250305 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-db66c2ef-40d4-4601-b835-5007abc591c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:43.250305 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:43.250305 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:43.250305 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:43.250305 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:43.250305 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:43.250305 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:43.250305 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:43.250305 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:43.250305 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:43.250305 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:43.250305 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:43.250305 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:43.250305 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:43.250305 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:43.250305 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:43.251316 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:43.251316 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:43.251316 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:43.251316 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:43.251316 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:43.251316 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:43.251316 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:43.251316 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:43.251316 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:43.251316 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:43.251316 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:43.251316 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:43.251316 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:43.251316 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:43.251316 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:43.251316 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:43.251316 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:43.252211 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:43.252211 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:43.252211 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:43.252211 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 515/1025] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:29:43.258443 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d3db7a-2515-4ab3-af80-8290668a821a None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:43.364554 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d3db7a-2515-4ab3-af80-8290668a821a None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:43.401038 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08df72b-2ec3-495e-b99f-678c4d757e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:43.401659 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08df72b-2ec3-495e-b99f-678c4d757e0c None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:43.401961 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08df72b-2ec3-495e-b99f-678c4d757e0c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:43.402229 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08df72b-2ec3-495e-b99f-678c4d757e0c None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:43.402882 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08df72b-2ec3-495e-b99f-678c4d757e0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:43.403177 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08df72b-2ec3-495e-b99f-678c4d757e0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:43.404565 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08df72b-2ec3-495e-b99f-678c4d757e0c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:43.406393 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:43.423340 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 511/1026] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:43.424661 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db66c2ef-40d4-4601-b835-5007abc591c4 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:43.526586 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db66c2ef-40d4-4601-b835-5007abc591c4 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:43.559836 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ba0b31-b2aa-4bdc-bd43-ffc404778588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:43.560457 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba0b31-b2aa-4bdc-bd43-ffc404778588 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:43.560735 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba0b31-b2aa-4bdc-bd43-ffc404778588 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:43.561003 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba0b31-b2aa-4bdc-bd43-ffc404778588 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:43.568731 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ba0b31-b2aa-4bdc-bd43-ffc404778588 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:43.570251 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ba0b31-b2aa-4bdc-bd43-ffc404778588 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:43.581213 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-60ba0b31-b2aa-4bdc-bd43-ffc404778588 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 28 15:29:43.581213 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:43.581213 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:43.581213 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:43.581213 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:43.581213 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:43.581213 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:43.581213 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:43.581213 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:43.581213 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:43.581213 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:43.581213 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:43.581213 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:29:43.581213 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:29:43.581213 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:29:43.581213 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:29:43.582451 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:43.582451 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:43.582451 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:29:43.582451 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:43.582451 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:29:43.582451 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:43.582451 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:43.582451 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:43.582451 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:43.582451 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:43.582451 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:43.582451 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:43.582451 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:43.582451 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:43.582451 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:43.582451 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:43.582451 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:43.582451 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:43.583578 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:43.583578 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:43.583578 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:29:43.583578 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:29:43.583578 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:29:43.583578 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:29:43.583578 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:29:43.583578 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:29:43.583578 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 28 15:29:43.583578 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:43.585997 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 516/1027] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Sep 28 15:29:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:43.588217 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08df72b-2ec3-495e-b99f-678c4d757e0c None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:43.689938 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08df72b-2ec3-495e-b99f-678c4d757e0c None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:43.724229 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2200e747-07bc-4c81-9ee5-18fcd8eeaeb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:43.724832 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2200e747-07bc-4c81-9ee5-18fcd8eeaeb3 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:43.725102 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2200e747-07bc-4c81-9ee5-18fcd8eeaeb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:43.725365 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2200e747-07bc-4c81-9ee5-18fcd8eeaeb3 None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:43.726105 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2200e747-07bc-4c81-9ee5-18fcd8eeaeb3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:43.726408 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2200e747-07bc-4c81-9ee5-18fcd8eeaeb3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:43.727807 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2200e747-07bc-4c81-9ee5-18fcd8eeaeb3 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:43.750138 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 512/1028] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Sep 28 15:29:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:43.751312 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ba0b31-b2aa-4bdc-bd43-ffc404778588 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:43.801328 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:43.853686 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ba0b31-b2aa-4bdc-bd43-ffc404778588 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:43.887052 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6c9cff-edd3-4aa7-8932-18b160655b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:43.887674 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c9cff-edd3-4aa7-8932-18b160655b2f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:43.887957 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c9cff-edd3-4aa7-8932-18b160655b2f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:43.888229 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c9cff-edd3-4aa7-8932-18b160655b2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:43.953554 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6c9cff-edd3-4aa7-8932-18b160655b2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:43.955026 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6c9cff-edd3-4aa7-8932-18b160655b2f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:44.002122 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:44.059634 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 517/1029] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:29:43 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 15:29:44.061103 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2200e747-07bc-4c81-9ee5-18fcd8eeaeb3 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:44.114107 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:44.166035 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2200e747-07bc-4c81-9ee5-18fcd8eeaeb3 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:44.201297 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f92503d-b268-4682-ae17-fa65728caf30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:44.201955 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f92503d-b268-4682-ae17-fa65728caf30 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:44.202236 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f92503d-b268-4682-ae17-fa65728caf30 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:44.202509 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f92503d-b268-4682-ae17-fa65728caf30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:44.212518 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f92503d-b268-4682-ae17-fa65728caf30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:44.213972 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f92503d-b268-4682-ae17-fa65728caf30 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:44.228780 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-9f92503d-b268-4682-ae17-fa65728caf30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:44.228780 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:44.228780 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:44.228780 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:44.228780 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:44.228780 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:44.228780 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:44.228780 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:44.228780 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:44.228780 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:44.228780 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:44.228780 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:44.228780 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:44.228780 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:44.228780 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:44.228780 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:44.229741 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:44.229741 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:44.229741 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:44.229741 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:44.229741 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:44.229741 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:44.229741 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:44.229741 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:44.229741 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:44.229741 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:44.229741 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:44.229741 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:44.229741 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:44.229741 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:44.229741 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:44.229741 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:44.229741 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:44.235771 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:44.235771 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:44.235771 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:44.235771 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 513/1030] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:29:44.236059 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6c9cff-edd3-4aa7-8932-18b160655b2f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:44.345689 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6c9cff-edd3-4aa7-8932-18b160655b2f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:44.380597 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf212f1-f616-4369-961a-885526d00c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:44.381121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf212f1-f616-4369-961a-885526d00c3e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:44.381400 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf212f1-f616-4369-961a-885526d00c3e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:44.381673 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf212f1-f616-4369-961a-885526d00c3e None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:44.382326 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf212f1-f616-4369-961a-885526d00c3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:44.382627 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf212f1-f616-4369-961a-885526d00c3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:44.383992 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf212f1-f616-4369-961a-885526d00c3e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:44.400311 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 518/1031] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:44.401925 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f92503d-b268-4682-ae17-fa65728caf30 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:44.508200 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f92503d-b268-4682-ae17-fa65728caf30 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:44.510557 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:44.544732 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0838efd3-bfdb-402f-84b7-ea2458f5d069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:44.545357 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0838efd3-bfdb-402f-84b7-ea2458f5d069 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:44.545633 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0838efd3-bfdb-402f-84b7-ea2458f5d069 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:44.545931 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0838efd3-bfdb-402f-84b7-ea2458f5d069 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:44.548681 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41667130-f92c-4165-a095-550161e63343 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:44.549007 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41667130-f92c-4165-a095-550161e63343 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:44.549328 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41667130-f92c-4165-a095-550161e63343 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:44.550967 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 519/1032] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:29:44.553550 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0838efd3-bfdb-402f-84b7-ea2458f5d069 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:44.555187 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0838efd3-bfdb-402f-84b7-ea2458f5d069 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:44.556414 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821598b-3d9e-4e1e-bfe0-d3e9bde3b2db None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:44.556698 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821598b-3d9e-4e1e-bfe0-d3e9bde3b2db None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:44.556967 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821598b-3d9e-4e1e-bfe0-d3e9bde3b2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:44.569496 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-0838efd3-bfdb-402f-84b7-ea2458f5d069 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:44.569496 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:44.569496 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:44.569496 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:44.569496 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:44.569496 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:44.569496 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:44.569496 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:44.569496 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:44.569496 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:44.569496 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:44.569496 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:44.569496 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:29:44.569496 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:29:44.569496 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:29:44.570567 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:29:44.570567 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:44.570567 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:44.570567 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:44.570567 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:44.570567 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:44.570567 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:44.570567 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:44.570567 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:44.570567 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:44.570567 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:44.570567 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:44.570567 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:44.570567 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:44.570567 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:44.570567 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:44.570567 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:44.571464 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:29:44.571464 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:29:44.571464 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:29:44.571464 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:29:44.571464 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:29:44.571464 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:29:44.571464 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:44.571464 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:44.572192 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 514/1033] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:29:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:44.578084 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0838efd3-bfdb-402f-84b7-ea2458f5d069 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:44.582397 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-6821598b-3d9e-4e1e-bfe0-d3e9bde3b2db None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:44.584257 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6821598b-3d9e-4e1e-bfe0-d3e9bde3b2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:44.586834 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:44.663851 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 520/1034] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:44 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 15:29:44.682067 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d19f7-0c6e-40a9-91d7-54f2e92b570e None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:44.682348 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d19f7-0c6e-40a9-91d7-54f2e92b570e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:44.682635 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d19f7-0c6e-40a9-91d7-54f2e92b570e None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:44.684188 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 521/1035] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:29:44.689672 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e915be9d-8900-4530-b233-6bd09193bc6b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:44.690257 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e915be9d-8900-4530-b233-6bd09193bc6b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:44.690533 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e915be9d-8900-4530-b233-6bd09193bc6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:44.691856 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0838efd3-bfdb-402f-84b7-ea2458f5d069 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:44.713566 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-e915be9d-8900-4530-b233-6bd09193bc6b None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:44.715321 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e915be9d-8900-4530-b233-6bd09193bc6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:44.728776 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a916c3-296b-4092-ad05-8000038858bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:44.729414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a916c3-296b-4092-ad05-8000038858bf None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:44.729702 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a916c3-296b-4092-ad05-8000038858bf None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:44.730003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a916c3-296b-4092-ad05-8000038858bf None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:44.730686 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a916c3-296b-4092-ad05-8000038858bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:44.730995 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a916c3-296b-4092-ad05-8000038858bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:44.732388 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a916c3-296b-4092-ad05-8000038858bf None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:44.749703 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 515/1036] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 15:29:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:44.751356 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a916c3-296b-4092-ad05-8000038858bf None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:44.778484 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:44.790618 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 522/1037] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:44 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:29:44.855766 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a916c3-296b-4092-ad05-8000038858bf None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:44.890467 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05a3e78-ab70-40d5-b478-18c9a7176722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:44.891350 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a3e78-ab70-40d5-b478-18c9a7176722 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:44.891773 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a3e78-ab70-40d5-b478-18c9a7176722 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:44.892201 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a3e78-ab70-40d5-b478-18c9a7176722 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:44.893067 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05a3e78-ab70-40d5-b478-18c9a7176722 None admin] RBAC: Adding query filter params (role.id=769ffe2b49154917b28e916692c53d5f, scope.project.id=3dc6c5f5fcd34922b0d09cd2aaed6cf1, user.id=8daf14b64bf04b2ba74d2fe1fd41ff71) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:44.893521 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05a3e78-ab70-40d5-b478-18c9a7176722 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:44.895123 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05a3e78-ab70-40d5-b478-18c9a7176722 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:44.901473 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 516/1038] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 15:29:44 2022] GET /identity/v3/role_assignments?user.id=8daf14b64bf04b2ba74d2fe1fd41ff71&scope.project.id=3dc6c5f5fcd34922b0d09cd2aaed6cf1&role.id=769ffe2b49154917b28e916692c53d5f => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:44.928965 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e915be9d-8900-4530-b233-6bd09193bc6b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:45.002440 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:45.042239 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e915be9d-8900-4530-b233-6bd09193bc6b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:45.060323 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05a3e78-ab70-40d5-b478-18c9a7176722 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:45.081043 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8de13de-3834-48a2-a329-5b8292879925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:45.081938 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de13de-3834-48a2-a329-5b8292879925 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:45.082384 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de13de-3834-48a2-a329-5b8292879925 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:45.082817 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de13de-3834-48a2-a329-5b8292879925 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:45.084111 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8de13de-3834-48a2-a329-5b8292879925 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:45.085548 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8de13de-3834-48a2-a329-5b8292879925 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:45.093873 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 523/1039] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:45.093873 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8de13de-3834-48a2-a329-5b8292879925 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:45.166646 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:45.180566 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05a3e78-ab70-40d5-b478-18c9a7176722 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:45.200368 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8de13de-3834-48a2-a329-5b8292879925 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:45.218195 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e66baa7-5465-4670-8667-3045e39a7e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:45.218822 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e66baa7-5465-4670-8667-3045e39a7e39 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:45.219100 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e66baa7-5465-4670-8667-3045e39a7e39 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:45.219370 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e66baa7-5465-4670-8667-3045e39a7e39 None admin] PATH_INFO: `/v3/services/image` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:45.219945 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e66baa7-5465-4670-8667-3045e39a7e39 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:45.221300 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e66baa7-5465-4670-8667-3045e39a7e39 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:45.230167 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-2e66baa7-5465-4670-8667-3045e39a7e39 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 28 15:29:45.230167 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:45.230167 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:29:45.230167 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:29:45.230167 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:29:45.230167 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:29:45.230167 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:29:45.230167 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:29:45.230167 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 28 15:29:45.230167 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:45.230167 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:29:45.230167 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:45.230167 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:45.230167 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:45.230167 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:45.230167 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:45.230167 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:45.231181 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:45.231181 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:45.231181 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:45.231181 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:45.231181 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:45.231181 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:45.231181 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:29:45.231181 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:29:45.231181 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:29:45.231181 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:29:45.231181 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:45.231181 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:45.231181 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:45.231181 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:45.231181 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:45.231181 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:45.232785 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:45.232785 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:45.232785 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:45.232785 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:45.232785 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:45.232785 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:45.232785 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:45.232785 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:45.232785 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:45.232785 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:45.232785 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:29:45.232785 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:29:45.232785 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 28 15:29:45.232785 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:45.232785 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 517/1040] 10.222.0.52 () {56 vars in 1105 bytes} [Wed Sep 28 15:29:45 2022] GET /identity/v3/services/image => generated 86 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:29:45.237100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73fbe73-1feb-4433-af52-4336358a6911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:45.237229 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e66baa7-5465-4670-8667-3045e39a7e39 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:45.237722 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73fbe73-1feb-4433-af52-4336358a6911 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:45.238041 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73fbe73-1feb-4433-af52-4336358a6911 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:45.238319 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73fbe73-1feb-4433-af52-4336358a6911 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:45.238965 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73fbe73-1feb-4433-af52-4336358a6911 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:45.240928 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73fbe73-1feb-4433-af52-4336358a6911 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:45.251145 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:45.277880 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 524/1041] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 15:29:45 2022] POST /identity/v3/users => generated 285 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:29:45.353904 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e66baa7-5465-4670-8667-3045e39a7e39 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:45.393517 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ffd2e2-a2f6-4010-a8e4-86ea07bb021b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:45.394173 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ffd2e2-a2f6-4010-a8e4-86ea07bb021b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:45.394461 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ffd2e2-a2f6-4010-a8e4-86ea07bb021b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:45.394782 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ffd2e2-a2f6-4010-a8e4-86ea07bb021b None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:45.395429 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ffd2e2-a2f6-4010-a8e4-86ea07bb021b None admin] RBAC: Adding query filter params (name=image) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:45.395858 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ffd2e2-a2f6-4010-a8e4-86ea07bb021b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:45.397278 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ffd2e2-a2f6-4010-a8e4-86ea07bb021b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:45.407442 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 518/1042] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:45 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:45.408846 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73fbe73-1feb-4433-af52-4336358a6911 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:45.516074 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73fbe73-1feb-4433-af52-4336358a6911 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:45.550549 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca73d3ad-6054-4889-884c-3d74d327beef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:45.551161 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73d3ad-6054-4889-884c-3d74d327beef None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:45.551444 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73d3ad-6054-4889-884c-3d74d327beef None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:45.551716 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca73d3ad-6054-4889-884c-3d74d327beef None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:45.552941 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca73d3ad-6054-4889-884c-3d74d327beef None admin] RBAC: Adding query filter params (type=image) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:45.553246 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca73d3ad-6054-4889-884c-3d74d327beef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:45.554653 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca73d3ad-6054-4889-884c-3d74d327beef None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:45.559704 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 525/1043] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:45 2022] GET /identity/v3/services?type=image => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:45.565399 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ffd2e2-a2f6-4010-a8e4-86ea07bb021b None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:45.584730 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:45.669290 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ffd2e2-a2f6-4010-a8e4-86ea07bb021b None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:45.702608 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5632d595-d37e-4407-b8ef-0f73bcad9d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:45.703211 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632d595-d37e-4407-b8ef-0f73bcad9d81 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:45.703485 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632d595-d37e-4407-b8ef-0f73bcad9d81 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:45.703754 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5632d595-d37e-4407-b8ef-0f73bcad9d81 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:45.704494 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5632d595-d37e-4407-b8ef-0f73bcad9d81 None admin] RBAC: Adding query filter params (interface=public, service_id=d6e1bb4e5d564aecb877c4a276c1db7d, region_id=RegionOne) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:45.704805 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5632d595-d37e-4407-b8ef-0f73bcad9d81 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:45.706184 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5632d595-d37e-4407-b8ef-0f73bcad9d81 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:45.711405 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 519/1044] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Sep 28 15:29:45 2022] GET /identity/v3/endpoints?service_id=d6e1bb4e5d564aecb877c4a276c1db7d&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:45.717186 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca73d3ad-6054-4889-884c-3d74d327beef None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:45.777484 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:45.824786 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca73d3ad-6054-4889-884c-3d74d327beef None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:45.856990 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c32e63-3b58-460c-b5b9-cbe0cedf700a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:45.857597 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c32e63-3b58-460c-b5b9-cbe0cedf700a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:45.857888 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c32e63-3b58-460c-b5b9-cbe0cedf700a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:45.858160 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c32e63-3b58-460c-b5b9-cbe0cedf700a None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:45.858748 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c32e63-3b58-460c-b5b9-cbe0cedf700a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:45.860135 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c32e63-3b58-460c-b5b9-cbe0cedf700a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:45.867271 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 526/1045] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Sep 28 15:29:45 2022] GET /identity/v3/services => generated 1957 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:29:47.045761 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75c8eb-308b-4a87-a160-e19d601ac51a None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:47.046307 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75c8eb-308b-4a87-a160-e19d601ac51a None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:47.046307 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75c8eb-308b-4a87-a160-e19d601ac51a None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:47.047781 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 520/1046] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:47.055007 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826df5fa-4ae9-4cde-8fcf-5e2048c42e90 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:47.055461 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826df5fa-4ae9-4cde-8fcf-5e2048c42e90 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:47.055538 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826df5fa-4ae9-4cde-8fcf-5e2048c42e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:47.074845 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-826df5fa-4ae9-4cde-8fcf-5e2048c42e90 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:47.076019 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-826df5fa-4ae9-4cde-8fcf-5e2048c42e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:47.144979 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 527/1047] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:47 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:29:47.412119 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b75c8eb-308b-4a87-a160-e19d601ac51a None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:47.487634 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04306bb-d9aa-40a6-a64f-9e220cb1afed None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:47.487978 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04306bb-d9aa-40a6-a64f-9e220cb1afed None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:47.488189 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04306bb-d9aa-40a6-a64f-9e220cb1afed None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:47.490317 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 528/1048] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:29:47.495067 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2c981-0913-46c1-82ad-ff72c7f81171 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:47.495350 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2c981-0913-46c1-82ad-ff72c7f81171 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:47.495631 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2c981-0913-46c1-82ad-ff72c7f81171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:47.498774 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:47.514742 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-5da2c981-0913-46c1-82ad-ff72c7f81171 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:47.515912 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5da2c981-0913-46c1-82ad-ff72c7f81171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:47.521642 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b75c8eb-308b-4a87-a160-e19d601ac51a None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:47.557162 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327d161-4c18-4eb3-b032-1ab70e44548a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:47.557790 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327d161-4c18-4eb3-b032-1ab70e44548a None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:47.558096 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327d161-4c18-4eb3-b032-1ab70e44548a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:47.558786 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327d161-4c18-4eb3-b032-1ab70e44548a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:47.564175 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:47.571145 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9327d161-4c18-4eb3-b032-1ab70e44548a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:47.575999 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-9327d161-4c18-4eb3-b032-1ab70e44548a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 15:29:47.575999 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:47.575999 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:47.575999 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:47.575999 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:47.575999 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:47.575999 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:47.575999 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:47.575999 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:47.575999 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:47.575999 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:47.575999 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:47.575999 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:47.575999 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:47.575999 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:47.575999 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:47.576969 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:47.576969 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:47.576969 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:47.576969 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:47.576969 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:47.576969 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:47.576969 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:47.576969 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:47.576969 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:47.576969 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:47.576969 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:47.576969 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:47.576969 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:47.576969 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:47.576969 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:47.576969 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:47.577810 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:47.577810 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:47.577810 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:47.577810 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:47.577810 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:47.577810 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:47.577810 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:47.577810 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:47.577810 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:47.577810 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:47.577810 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 15:29:47.577810 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:47.577810 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 521/1049] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Sep 28 15:29:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:47.583012 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327d161-4c18-4eb3-b032-1ab70e44548a None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:47.591492 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 529/1050] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:47 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 15:29:47.694024 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327d161-4c18-4eb3-b032-1ab70e44548a None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:47.728427 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf5d653-e5cb-4114-af9a-b0f3986f6e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:47.730785 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5d653-e5cb-4114-af9a-b0f3986f6e0d None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:47.730785 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5d653-e5cb-4114-af9a-b0f3986f6e0d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:47.730785 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5d653-e5cb-4114-af9a-b0f3986f6e0d None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:47.730785 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf5d653-e5cb-4114-af9a-b0f3986f6e0d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:47.730785 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf5d653-e5cb-4114-af9a-b0f3986f6e0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:47.732265 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf5d653-e5cb-4114-af9a-b0f3986f6e0d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:47.746052 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 522/1051] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Sep 28 15:29:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:47.747332 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da2c981-0913-46c1-82ad-ff72c7f81171 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:47.853373 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da2c981-0913-46c1-82ad-ff72c7f81171 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:47.868677 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf5d653-e5cb-4114-af9a-b0f3986f6e0d None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:47.871583 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:47.890058 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384a9348-6ba5-4d6c-b3cc-639cb0c21064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:47.890715 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a9348-6ba5-4d6c-b3cc-639cb0c21064 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:47.890992 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a9348-6ba5-4d6c-b3cc-639cb0c21064 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:47.891259 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a9348-6ba5-4d6c-b3cc-639cb0c21064 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:47.902892 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384a9348-6ba5-4d6c-b3cc-639cb0c21064 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:47.904377 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384a9348-6ba5-4d6c-b3cc-639cb0c21064 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:47.920488 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-384a9348-6ba5-4d6c-b3cc-639cb0c21064 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:47.920488 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:47.920488 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:47.920488 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:47.920488 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:47.920488 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:47.920488 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:47.920488 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:47.920488 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:47.920488 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:47.920488 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:47.920488 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:47.920488 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:47.920488 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:47.920488 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:47.920488 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:47.921507 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:47.921507 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:47.921507 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:47.921507 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:47.921507 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:47.921507 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:47.921507 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:47.921507 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:47.921507 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:47.921507 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:47.921507 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:47.921507 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:47.921507 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:47.921507 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:47.921507 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:47.921507 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:47.921507 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:47.922797 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:47.922797 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:47.922797 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:47.922797 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 530/1052] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:47.927362 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384a9348-6ba5-4d6c-b3cc-639cb0c21064 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:47.958860 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:47.992445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf5d653-e5cb-4114-af9a-b0f3986f6e0d None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:48.032792 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f604e217-c6f8-46ea-8e54-ce18ed2d36f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:48.033412 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f604e217-c6f8-46ea-8e54-ce18ed2d36f0 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:48.033685 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f604e217-c6f8-46ea-8e54-ce18ed2d36f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:48.033974 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f604e217-c6f8-46ea-8e54-ce18ed2d36f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:48.034102 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384a9348-6ba5-4d6c-b3cc-639cb0c21064 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:48.045446 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f604e217-c6f8-46ea-8e54-ce18ed2d36f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:48.050223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-f604e217-c6f8-46ea-8e54-ce18ed2d36f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:29:48.050223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:48.050223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:48.050223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:48.050223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:48.050223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:48.050223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:48.050223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:48.050223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:48.050223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:48.050223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:48.050223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:48.050223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:48.050223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:48.050223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:48.050223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:48.051946 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:48.051946 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:48.051946 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:48.051946 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:48.051946 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:48.051946 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:48.051946 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:48.051946 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:48.051946 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:48.051946 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:48.051946 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:48.051946 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:48.051946 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:48.051946 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:48.051946 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:48.051946 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:48.053020 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:48.053020 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:48.053020 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:48.053020 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:48.053020 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:48.053020 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:48.053020 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:48.053020 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:48.053020 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:48.053020 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:48.053020 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:29:48.053020 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:48.053020 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 523/1053] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 15:29:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:29:48.053020 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af7b1e-e0d0-4408-bb87-05337efce8af None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:48.053785 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af7b1e-e0d0-4408-bb87-05337efce8af None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:48.053785 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af7b1e-e0d0-4408-bb87-05337efce8af None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:48.054473 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 524/1054] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:48.057077 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82af7b1e-e0d0-4408-bb87-05337efce8af None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:48.070194 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d011b-9041-4ae2-a960-b256518d3b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:48.070813 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d011b-9041-4ae2-a960-b256518d3b6a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:48.071096 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d011b-9041-4ae2-a960-b256518d3b6a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:48.071394 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d011b-9041-4ae2-a960-b256518d3b6a None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:48.072031 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d011b-9041-4ae2-a960-b256518d3b6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:48.072332 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d011b-9041-4ae2-a960-b256518d3b6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:48.073709 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d011b-9041-4ae2-a960-b256518d3b6a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:48.086259 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 531/1055] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:48.087446 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bceb45-b590-4f13-a15a-152fc92319e8 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:48.087734 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bceb45-b590-4f13-a15a-152fc92319e8 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:48.088008 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bceb45-b590-4f13-a15a-152fc92319e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:48.107602 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-34bceb45-b590-4f13-a15a-152fc92319e8 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:48.108814 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-34bceb45-b590-4f13-a15a-152fc92319e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:48.132626 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:48.176724 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82af7b1e-e0d0-4408-bb87-05337efce8af None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:48.180147 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 532/1056] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:48 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:29:48.180982 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bceb45-b590-4f13-a15a-152fc92319e8 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:48.208420 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:48.212871 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2bf7af-b363-44a1-b0b9-f6cc85a827d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:48.213407 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2bf7af-b363-44a1-b0b9-f6cc85a827d7 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:48.213686 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2bf7af-b363-44a1-b0b9-f6cc85a827d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:48.213978 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2bf7af-b363-44a1-b0b9-f6cc85a827d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:48.214634 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2bf7af-b363-44a1-b0b9-f6cc85a827d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:48.214933 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2bf7af-b363-44a1-b0b9-f6cc85a827d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:48.216278 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2bf7af-b363-44a1-b0b9-f6cc85a827d7 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:48.225041 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 525/1057] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:29:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:48.230876 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2bf7af-b363-44a1-b0b9-f6cc85a827d7 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:48.297043 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bceb45-b590-4f13-a15a-152fc92319e8 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:48.334668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1febb596-3a6a-44cb-a31c-282eabbcfc34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:48.335290 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1febb596-3a6a-44cb-a31c-282eabbcfc34 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:48.335568 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1febb596-3a6a-44cb-a31c-282eabbcfc34 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:48.335828 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1febb596-3a6a-44cb-a31c-282eabbcfc34 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:48.342357 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2bf7af-b363-44a1-b0b9-f6cc85a827d7 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:48.342879 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1febb596-3a6a-44cb-a31c-282eabbcfc34 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:48.344313 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1febb596-3a6a-44cb-a31c-282eabbcfc34 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:48.356607 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-1febb596-3a6a-44cb-a31c-282eabbcfc34 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 28 15:29:48.356607 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:48.356607 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:48.356607 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:48.356607 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:48.356607 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:48.356607 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:48.356607 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:48.356607 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:48.356607 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:48.356607 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:48.356607 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:48.356607 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:29:48.356607 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:29:48.356607 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:29:48.356607 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:29:48.362391 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:48.362391 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:48.362391 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:29:48.362391 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:48.362391 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:29:48.362391 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:48.362391 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:48.362391 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:48.362391 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:48.362391 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:48.362391 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:48.362391 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:48.362391 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:48.362391 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:48.362391 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:48.362391 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:48.362391 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:48.362391 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:48.363609 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:48.363609 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:48.363609 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:29:48.363609 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:29:48.363609 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:29:48.363609 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:29:48.363609 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:29:48.363609 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:29:48.363609 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 28 15:29:48.363609 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:48.363609 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 533/1058] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Sep 28 15:29:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:48.363609 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1febb596-3a6a-44cb-a31c-282eabbcfc34 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:48.369585 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:48.381960 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc73454-8b47-4d8f-8243-73fa93d89c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:48.382704 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc73454-8b47-4d8f-8243-73fa93d89c8e None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:48.383096 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc73454-8b47-4d8f-8243-73fa93d89c8e None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:48.383465 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc73454-8b47-4d8f-8243-73fa93d89c8e None admin] PATH_INFO: `/v3/users/4cc5e55c56fa46f4b5432867d44a7005` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:48.397277 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc73454-8b47-4d8f-8243-73fa93d89c8e None admin] RBAC: Authorizing `identity:get_user(user_id=4cc5e55c56fa46f4b5432867d44a7005)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:48.398840 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc73454-8b47-4d8f-8243-73fa93d89c8e None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:48.405176 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 526/1059] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:29:48 2022] GET /identity/v3/users/4cc5e55c56fa46f4b5432867d44a7005 => generated 285 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:48.406043 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc73454-8b47-4d8f-8243-73fa93d89c8e None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:48.457184 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:48.475868 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1febb596-3a6a-44cb-a31c-282eabbcfc34 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:48.508673 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a176392-aa00-49ad-af7f-12839310acd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:48.509204 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc73454-8b47-4d8f-8243-73fa93d89c8e None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:48.509305 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a176392-aa00-49ad-af7f-12839310acd2 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:48.509536 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a176392-aa00-49ad-af7f-12839310acd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:48.509808 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a176392-aa00-49ad-af7f-12839310acd2 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:48.510553 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a176392-aa00-49ad-af7f-12839310acd2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:48.510854 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a176392-aa00-49ad-af7f-12839310acd2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:48.512335 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a176392-aa00-49ad-af7f-12839310acd2 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:48.528815 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 534/1060] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Sep 28 15:29:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:48.528815 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a176392-aa00-49ad-af7f-12839310acd2 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:48.544016 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88801134-1c9a-453f-9715-e4f05ef0e32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:48.544620 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88801134-1c9a-453f-9715-e4f05ef0e32d None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:48.544896 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88801134-1c9a-453f-9715-e4f05ef0e32d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:48.545167 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88801134-1c9a-453f-9715-e4f05ef0e32d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:48.592619 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:48.613324 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88801134-1c9a-453f-9715-e4f05ef0e32d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:48.615339 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88801134-1c9a-453f-9715-e4f05ef0e32d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:48.627027 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a176392-aa00-49ad-af7f-12839310acd2 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:48.659330 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27ef0a0-a1d5-477d-935d-d37ffd1cba11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:48.659949 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ef0a0-a1d5-477d-935d-d37ffd1cba11 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:48.660224 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ef0a0-a1d5-477d-935d-d37ffd1cba11 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:48.660490 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ef0a0-a1d5-477d-935d-d37ffd1cba11 None admin] PATH_INFO: `/v3/services/image` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:48.661607 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27ef0a0-a1d5-477d-935d-d37ffd1cba11 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:48.662988 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27ef0a0-a1d5-477d-935d-d37ffd1cba11 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:48.671340 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-a27ef0a0-a1d5-477d-935d-d37ffd1cba11 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 28 15:29:48.671340 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:48.671340 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:29:48.671340 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:29:48.671340 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:29:48.671340 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:29:48.671340 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:29:48.671340 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:29:48.671340 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 28 15:29:48.671340 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:48.671340 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:29:48.671340 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:48.671340 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:48.671340 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:48.671340 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:48.671340 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:48.671340 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:48.672470 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:48.672470 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:48.672470 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:48.672470 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:48.672470 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:48.672470 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:48.672470 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:29:48.672470 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:29:48.672470 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:29:48.672470 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:29:48.672470 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:48.672470 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:48.672470 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:48.672470 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:48.672470 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:48.672470 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:48.673319 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:48.673319 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:48.673319 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:48.673319 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:48.673319 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:48.673319 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:48.673319 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:48.673319 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:48.673319 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:48.673319 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:48.673319 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:29:48.673319 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:29:48.673319 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 28 15:29:48.673319 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:48.673319 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 535/1061] 10.222.0.52 () {56 vars in 1105 bytes} [Wed Sep 28 15:29:48 2022] GET /identity/v3/services/image => generated 86 bytes in 143 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:48.674327 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27ef0a0-a1d5-477d-935d-d37ffd1cba11 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:48.680778 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:48.713713 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 527/1062] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:29:48 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 28 15:29:48.714718 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88801134-1c9a-453f-9715-e4f05ef0e32d None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:48.776213 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27ef0a0-a1d5-477d-935d-d37ffd1cba11 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:48.794380 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:48.809246 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54741b17-f902-4e1f-bd39-a75abcdb8e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:48.809879 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54741b17-f902-4e1f-bd39-a75abcdb8e22 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:48.810164 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54741b17-f902-4e1f-bd39-a75abcdb8e22 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:48.810432 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54741b17-f902-4e1f-bd39-a75abcdb8e22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:48.813049 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88801134-1c9a-453f-9715-e4f05ef0e32d None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:48.821073 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54741b17-f902-4e1f-bd39-a75abcdb8e22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:48.822539 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54741b17-f902-4e1f-bd39-a75abcdb8e22 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:48.836339 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-54741b17-f902-4e1f-bd39-a75abcdb8e22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:48.836339 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:48.836339 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:48.836339 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:48.836339 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:48.836339 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:48.836339 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:48.836339 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:48.836339 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:48.836339 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:48.836339 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:48.836339 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:48.836339 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:48.836339 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:48.836339 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:48.836339 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:48.837463 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:48.837463 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:48.837463 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:48.837463 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:48.837463 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:48.837463 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:48.837463 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:48.837463 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:48.837463 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:48.837463 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:48.837463 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:48.837463 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:48.837463 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:48.837463 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:48.837463 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:48.837463 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:48.837463 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:48.838618 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:48.838618 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:48.838618 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:48.838618 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 536/1063] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:48.838618 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54741b17-f902-4e1f-bd39-a75abcdb8e22 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:48.851777 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72255041-e1f3-42cd-9a9d-eae933ade655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:48.852377 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72255041-e1f3-42cd-9a9d-eae933ade655 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:48.852646 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72255041-e1f3-42cd-9a9d-eae933ade655 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:48.852914 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72255041-e1f3-42cd-9a9d-eae933ade655 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:48.853527 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72255041-e1f3-42cd-9a9d-eae933ade655 None admin] RBAC: Adding query filter params (name=image) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:48.853823 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72255041-e1f3-42cd-9a9d-eae933ade655 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:48.855175 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72255041-e1f3-42cd-9a9d-eae933ade655 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:48.865119 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 528/1064] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:48 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:48.865119 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72255041-e1f3-42cd-9a9d-eae933ade655 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:48.900707 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:48.944580 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54741b17-f902-4e1f-bd39-a75abcdb8e22 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:48.958792 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72255041-e1f3-42cd-9a9d-eae933ade655 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:48.976804 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9b8815-e356-4a9b-982b-70725ab5d6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:48.977518 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b8815-e356-4a9b-982b-70725ab5d6e3 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:48.977799 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b8815-e356-4a9b-982b-70725ab5d6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:48.978111 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b8815-e356-4a9b-982b-70725ab5d6e3 None admin] PATH_INFO: `/v3/projects/e1e943c8e9924e0bab7d96b25ee1fd5b` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:48.979255 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9b8815-e356-4a9b-982b-70725ab5d6e3 None admin] RBAC: Authorizing `identity:get_project(project_id=e1e943c8e9924e0bab7d96b25ee1fd5b)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:48.980778 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9b8815-e356-4a9b-982b-70725ab5d6e3 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:48.982501 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 537/1065] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:29:48 2022] GET /identity/v3/projects/e1e943c8e9924e0bab7d96b25ee1fd5b => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:48.983257 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9b8815-e356-4a9b-982b-70725ab5d6e3 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:48.991974 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d84fb-fe32-4f56-9916-f0040378eac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:48.992575 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d84fb-fe32-4f56-9916-f0040378eac1 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:48.992851 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d84fb-fe32-4f56-9916-f0040378eac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:48.993119 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d84fb-fe32-4f56-9916-f0040378eac1 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:48.993747 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045d84fb-fe32-4f56-9916-f0040378eac1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:48.994075 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045d84fb-fe32-4f56-9916-f0040378eac1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:48.995417 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045d84fb-fe32-4f56-9916-f0040378eac1 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:49.008400 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 529/1066] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:49.009173 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d84fb-fe32-4f56-9916-f0040378eac1 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:49.018934 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:49.085342 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9b8815-e356-4a9b-982b-70725ab5d6e3 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:49.107843 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:49.107972 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d84fb-fe32-4f56-9916-f0040378eac1 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:49.119769 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006dd9ca-9764-44ac-b196-68862b13ccb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:49.120371 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006dd9ca-9764-44ac-b196-68862b13ccb2 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:49.120652 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006dd9ca-9764-44ac-b196-68862b13ccb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:49.120928 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006dd9ca-9764-44ac-b196-68862b13ccb2 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:49.121544 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006dd9ca-9764-44ac-b196-68862b13ccb2 None admin] RBAC: Adding query filter params (type=image) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:49.121867 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006dd9ca-9764-44ac-b196-68862b13ccb2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:49.123249 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006dd9ca-9764-44ac-b196-68862b13ccb2 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:49.128177 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 538/1067] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:48 2022] GET /identity/v3/services?type=image => generated 350 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:49.128788 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006dd9ca-9764-44ac-b196-68862b13ccb2 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:49.143092 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855cd54e-a194-4312-a06b-9d3c003ad58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:49.143694 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855cd54e-a194-4312-a06b-9d3c003ad58b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:49.143974 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855cd54e-a194-4312-a06b-9d3c003ad58b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:49.144241 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855cd54e-a194-4312-a06b-9d3c003ad58b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:49.144961 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855cd54e-a194-4312-a06b-9d3c003ad58b None admin] RBAC: Adding query filter params (role.id=62d96ac9513c452894fc67b442cfc59f, scope.project.id=e1e943c8e9924e0bab7d96b25ee1fd5b, user.id=4cc5e55c56fa46f4b5432867d44a7005) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:49.145266 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855cd54e-a194-4312-a06b-9d3c003ad58b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:49.146703 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855cd54e-a194-4312-a06b-9d3c003ad58b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:49.151916 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 530/1068] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 15:29:49 2022] GET /identity/v3/role_assignments?user.id=4cc5e55c56fa46f4b5432867d44a7005&scope.project.id=e1e943c8e9924e0bab7d96b25ee1fd5b&role.id=62d96ac9513c452894fc67b442cfc59f => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:49.152647 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855cd54e-a194-4312-a06b-9d3c003ad58b None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:49.234109 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:49.255580 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006dd9ca-9764-44ac-b196-68862b13ccb2 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:49.268988 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855cd54e-a194-4312-a06b-9d3c003ad58b None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:49.297970 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88a98dc-68c0-4d90-be0b-72bd26385167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:49.298615 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a98dc-68c0-4d90-be0b-72bd26385167 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:49.298894 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a98dc-68c0-4d90-be0b-72bd26385167 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:49.299164 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88a98dc-68c0-4d90-be0b-72bd26385167 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:49.307208 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88a98dc-68c0-4d90-be0b-72bd26385167 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:49.308680 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88a98dc-68c0-4d90-be0b-72bd26385167 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:49.309091 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b821e5-32e6-405a-9f75-19989b8f3864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:49.309705 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b821e5-32e6-405a-9f75-19989b8f3864 None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:49.310015 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b821e5-32e6-405a-9f75-19989b8f3864 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:49.310289 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b821e5-32e6-405a-9f75-19989b8f3864 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:49.310961 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b821e5-32e6-405a-9f75-19989b8f3864 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:49.312395 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b821e5-32e6-405a-9f75-19989b8f3864 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:49.321164 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-d88a98dc-68c0-4d90-be0b-72bd26385167 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:49.321164 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:49.321164 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:49.321164 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:49.321164 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:49.321164 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:49.321164 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:49.321164 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:49.321164 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:49.321164 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:49.321164 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:49.321164 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:49.321164 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:29:49.321164 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:29:49.321164 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:29:49.322159 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:29:49.322159 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:49.322159 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:49.322159 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:49.322159 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:49.322159 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:49.322159 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:49.322159 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:49.322159 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:49.322159 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:49.322159 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:49.322159 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:49.322159 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:49.322159 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:49.322159 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:49.322159 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:49.322159 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:49.323050 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:29:49.323050 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:29:49.323050 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:29:49.323050 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:29:49.323050 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:29:49.323050 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:29:49.323050 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:49.323050 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:49.323050 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 539/1069] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:29:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:49.328199 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88a98dc-68c0-4d90-be0b-72bd26385167 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:49.349966 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 531/1070] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:29:49 2022] POST /identity/v3/endpoints => generated 327 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:29:49.387120 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:49.444908 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88a98dc-68c0-4d90-be0b-72bd26385167 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:49.495202 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbc13d6-cc00-4ab9-9978-450d6440a60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:49.495907 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc13d6-cc00-4ab9-9978-450d6440a60d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:49.496253 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc13d6-cc00-4ab9-9978-450d6440a60d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:49.496584 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc13d6-cc00-4ab9-9978-450d6440a60d None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:49.497386 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbc13d6-cc00-4ab9-9978-450d6440a60d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:49.497752 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbc13d6-cc00-4ab9-9978-450d6440a60d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:49.499271 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbc13d6-cc00-4ab9-9978-450d6440a60d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:49.514272 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 540/1071] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 15:29:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:49.521053 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b821e5-32e6-405a-9f75-19989b8f3864 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:49.623988 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b821e5-32e6-405a-9f75-19989b8f3864 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:49.649983 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:49.659984 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb1bc74-37eb-4427-adb0-61aa0473933d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:49.660760 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1bc74-37eb-4427-adb0-61aa0473933d None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:49.661714 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1bc74-37eb-4427-adb0-61aa0473933d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:49.662103 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1bc74-37eb-4427-adb0-61aa0473933d None admin] PATH_INFO: `/v3/projects/3dc6c5f5fcd34922b0d09cd2aaed6cf1/users/8daf14b64bf04b2ba74d2fe1fd41ff71/roles/769ffe2b49154917b28e916692c53d5f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:49.664303 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb1bc74-37eb-4427-adb0-61aa0473933d None admin] RBAC: Authorizing `identity:create_grant(project_id=3dc6c5f5fcd34922b0d09cd2aaed6cf1, user_id=8daf14b64bf04b2ba74d2fe1fd41ff71, role_id=769ffe2b49154917b28e916692c53d5f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:49.665805 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb1bc74-37eb-4427-adb0-61aa0473933d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:49.685877 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 532/1072] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 15:29:49 2022] PUT /identity/v3/projects/3dc6c5f5fcd34922b0d09cd2aaed6cf1/users/8daf14b64bf04b2ba74d2fe1fd41ff71/roles/769ffe2b49154917b28e916692c53d5f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:29:51.356190 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78286ddf-b942-40f1-a976-6238e8418730 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:51.356759 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78286ddf-b942-40f1-a976-6238e8418730 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:51.356759 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78286ddf-b942-40f1-a976-6238e8418730 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:51.358238 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 541/1073] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:51.363321 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d38d9b-a2ff-48b3-bcd7-b5b33bb4576e None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:51.363726 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d38d9b-a2ff-48b3-bcd7-b5b33bb4576e None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:51.364489 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d38d9b-a2ff-48b3-bcd7-b5b33bb4576e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:51.383705 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-92d38d9b-a2ff-48b3-bcd7-b5b33bb4576e None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:51.385213 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-92d38d9b-a2ff-48b3-bcd7-b5b33bb4576e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:51.460127 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 533/1074] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:29:51.527450 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b7bf56-111c-4213-b0aa-befd72b696e1 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:51.527756 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b7bf56-111c-4213-b0aa-befd72b696e1 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:51.527996 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b7bf56-111c-4213-b0aa-befd72b696e1 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:51.529445 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 542/1075] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:51.534393 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7a483-d574-4719-a186-83ea48d73a54 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:51.534678 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7a483-d574-4719-a186-83ea48d73a54 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:51.534949 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7a483-d574-4719-a186-83ea48d73a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:51.554162 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-fde7a483-d574-4719-a186-83ea48d73a54 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:51.555339 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-fde7a483-d574-4719-a186-83ea48d73a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:51.620477 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 534/1076] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:29:51.728502 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b7bf56-111c-4213-b0aa-befd72b696e1 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:51.833109 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb42c7-afe8-4fc0-8c24-99f2d30b38da None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:51.833400 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb42c7-afe8-4fc0-8c24-99f2d30b38da None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:51.833703 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb42c7-afe8-4fc0-8c24-99f2d30b38da None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:51.835085 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 535/1077] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:29:51.835652 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b7bf56-111c-4213-b0aa-befd72b696e1 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:51.839129 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:51.840333 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75792c2f-a764-47fa-b881-3f7663ee1ea4 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:51.840627 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75792c2f-a764-47fa-b881-3f7663ee1ea4 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:51.840908 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75792c2f-a764-47fa-b881-3f7663ee1ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:51.860700 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-75792c2f-a764-47fa-b881-3f7663ee1ea4 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:51.861919 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-75792c2f-a764-47fa-b881-3f7663ee1ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:51.873891 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5b60d-990b-4a14-a1e2-b48b01f4d37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:51.874548 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5b60d-990b-4a14-a1e2-b48b01f4d37a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:51.874832 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5b60d-990b-4a14-a1e2-b48b01f4d37a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:51.875104 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5b60d-990b-4a14-a1e2-b48b01f4d37a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:51.886774 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a5b60d-990b-4a14-a1e2-b48b01f4d37a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:51.891994 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-10a5b60d-990b-4a14-a1e2-b48b01f4d37a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:29:51.891994 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:51.891994 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:51.891994 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:51.891994 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:51.891994 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:51.891994 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:51.891994 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:51.891994 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:51.891994 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:51.891994 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:51.891994 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:51.891994 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:51.891994 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:51.891994 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:51.891994 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:51.893000 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:51.893000 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:51.893000 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:51.893000 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:51.893000 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:51.893000 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:51.893000 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:51.893000 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:51.893000 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:51.893000 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:51.893000 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:51.893000 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:51.893000 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:51.893000 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:51.893000 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:51.893000 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:51.893936 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:51.893936 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:51.893936 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:51.893936 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:51.893936 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:51.893936 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:51.893936 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:51.893936 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:51.893936 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:51.893936 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:51.893936 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:29:51.893936 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:51.893936 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 543/1078] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 15:29:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:51.894693 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5b60d-990b-4a14-a1e2-b48b01f4d37a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:51.911832 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:51.936931 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 536/1079] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 15:29:51.937744 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75792c2f-a764-47fa-b881-3f7663ee1ea4 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:51.998915 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5b60d-990b-4a14-a1e2-b48b01f4d37a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:52.035197 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0085554-2ccd-4a9b-93aa-8d6487ae4dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:52.036352 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0085554-2ccd-4a9b-93aa-8d6487ae4dda None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:52.036637 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0085554-2ccd-4a9b-93aa-8d6487ae4dda None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:52.036909 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0085554-2ccd-4a9b-93aa-8d6487ae4dda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:52.044154 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75792c2f-a764-47fa-b881-3f7663ee1ea4 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:52.071030 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:52.078786 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0f5d0-91ac-43c8-abb1-ebf3c5ff340f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:52.079396 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0f5d0-91ac-43c8-abb1-ebf3c5ff340f None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:52.079677 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0f5d0-91ac-43c8-abb1-ebf3c5ff340f None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:52.079947 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0f5d0-91ac-43c8-abb1-ebf3c5ff340f None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:52.080603 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f0f5d0-91ac-43c8-abb1-ebf3c5ff340f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:52.081321 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f0f5d0-91ac-43c8-abb1-ebf3c5ff340f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:52.083720 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f0f5d0-91ac-43c8-abb1-ebf3c5ff340f None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:52.097029 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 537/1080] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:29:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:52.098097 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0f5d0-91ac-43c8-abb1-ebf3c5ff340f None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:52.114620 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0085554-2ccd-4a9b-93aa-8d6487ae4dda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:52.115785 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0085554-2ccd-4a9b-93aa-8d6487ae4dda None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:52.140261 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:52.206639 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0f5d0-91ac-43c8-abb1-ebf3c5ff340f None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:52.224430 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 544/1081] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:29:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 15:29:52.225133 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0085554-2ccd-4a9b-93aa-8d6487ae4dda None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:52.242514 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aa535d-7e16-47a8-93e5-69674b11bdf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:52.243137 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa535d-7e16-47a8-93e5-69674b11bdf3 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:52.243412 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa535d-7e16-47a8-93e5-69674b11bdf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:52.243682 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa535d-7e16-47a8-93e5-69674b11bdf3 None admin] PATH_INFO: `/v3/users/4cc5e55c56fa46f4b5432867d44a7005` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:52.244776 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aa535d-7e16-47a8-93e5-69674b11bdf3 None admin] RBAC: Authorizing `identity:get_user(user_id=4cc5e55c56fa46f4b5432867d44a7005)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:52.246332 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aa535d-7e16-47a8-93e5-69674b11bdf3 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:52.252539 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 538/1082] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:29:52 2022] GET /identity/v3/users/4cc5e55c56fa46f4b5432867d44a7005 => generated 285 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:52.252539 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aa535d-7e16-47a8-93e5-69674b11bdf3 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:52.297332 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:52.337018 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0085554-2ccd-4a9b-93aa-8d6487ae4dda None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:52.351097 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aa535d-7e16-47a8-93e5-69674b11bdf3 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:52.357938 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:52.371192 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54793f34-07b7-4b5c-88c1-d751b5d26500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:52.371822 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54793f34-07b7-4b5c-88c1-d751b5d26500 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:52.372102 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54793f34-07b7-4b5c-88c1-d751b5d26500 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:52.372376 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54793f34-07b7-4b5c-88c1-d751b5d26500 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:52.382421 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54793f34-07b7-4b5c-88c1-d751b5d26500 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:52.385454 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee24784-35a4-4047-97d1-105f9e78696b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:52.386097 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee24784-35a4-4047-97d1-105f9e78696b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:52.386372 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee24784-35a4-4047-97d1-105f9e78696b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:52.386638 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee24784-35a4-4047-97d1-105f9e78696b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:52.387238 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-54793f34-07b7-4b5c-88c1-d751b5d26500 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 15:29:52.387238 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:52.387238 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:52.387238 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:52.387238 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:52.387238 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:52.387238 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:52.387238 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:52.387238 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:52.387238 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:52.387238 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:52.387238 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:52.387238 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:52.387238 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:52.387238 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:52.387238 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:52.388136 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:52.388136 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:52.388136 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:52.388136 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:52.388136 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:52.388136 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:52.388136 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:52.388136 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:52.388136 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:52.388136 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:52.388136 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:52.388136 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:52.388136 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:52.388136 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:52.388136 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:52.388136 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:52.389107 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:52.389107 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:52.389107 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:52.389107 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:52.389107 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:52.389107 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:52.389107 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:52.389107 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:52.389107 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:52.389107 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:52.389107 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 28 15:29:52.389107 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:52.389107 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 545/1083] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Sep 28 15:29:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:52.389107 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54793f34-07b7-4b5c-88c1-d751b5d26500 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:52.398253 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee24784-35a4-4047-97d1-105f9e78696b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:52.399683 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee24784-35a4-4047-97d1-105f9e78696b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:52.414668 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-dee24784-35a4-4047-97d1-105f9e78696b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:52.414668 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:52.414668 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:52.414668 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:52.414668 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:52.414668 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:52.414668 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:52.414668 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:52.414668 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:52.414668 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:52.414668 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:52.414668 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:52.414668 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:52.414668 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:52.414668 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:52.414668 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:52.415625 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:52.415625 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:52.415625 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:52.415625 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:52.415625 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:52.415625 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:52.415625 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:52.415625 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:52.415625 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:52.415625 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:52.415625 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:52.415625 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:52.415625 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:52.415625 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:52.415625 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:52.415625 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:52.415625 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:52.416508 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:52.416508 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:52.416508 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:52.416508 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 539/1084] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:52.416508 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee24784-35a4-4047-97d1-105f9e78696b None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:52.491108 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54793f34-07b7-4b5c-88c1-d751b5d26500 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:52.514388 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:52.514712 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee24784-35a4-4047-97d1-105f9e78696b None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:52.526252 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89468e10-af82-475d-9a46-93d73f867821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:52.526872 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89468e10-af82-475d-9a46-93d73f867821 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:52.527149 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89468e10-af82-475d-9a46-93d73f867821 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:52.527415 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89468e10-af82-475d-9a46-93d73f867821 None admin] PATH_INFO: `/v3/projects/e1e943c8e9924e0bab7d96b25ee1fd5b` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:52.528507 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89468e10-af82-475d-9a46-93d73f867821 None admin] RBAC: Authorizing `identity:get_project(project_id=e1e943c8e9924e0bab7d96b25ee1fd5b)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:52.529964 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89468e10-af82-475d-9a46-93d73f867821 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:52.531564 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 546/1085] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:29:52 2022] GET /identity/v3/projects/e1e943c8e9924e0bab7d96b25ee1fd5b => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:52.532313 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89468e10-af82-475d-9a46-93d73f867821 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:52.550338 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8920ef-6f37-4169-82b9-b215cb6717e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:52.550932 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8920ef-6f37-4169-82b9-b215cb6717e8 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:52.551209 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8920ef-6f37-4169-82b9-b215cb6717e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:52.551509 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8920ef-6f37-4169-82b9-b215cb6717e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:52.552155 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8920ef-6f37-4169-82b9-b215cb6717e8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:52.552456 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8920ef-6f37-4169-82b9-b215cb6717e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:52.553782 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8920ef-6f37-4169-82b9-b215cb6717e8 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:52.561496 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 540/1086] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Sep 28 15:29:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:52.562303 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8920ef-6f37-4169-82b9-b215cb6717e8 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:52.586672 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:52.630843 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89468e10-af82-475d-9a46-93d73f867821 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:52.662177 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8920ef-6f37-4169-82b9-b215cb6717e8 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:52.664005 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9efaa1-c3aa-4699-be7c-cb2a1f129d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:52.664611 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9efaa1-c3aa-4699-be7c-cb2a1f129d79 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:52.664893 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9efaa1-c3aa-4699-be7c-cb2a1f129d79 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:52.665166 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9efaa1-c3aa-4699-be7c-cb2a1f129d79 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:52.665791 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9efaa1-c3aa-4699-be7c-cb2a1f129d79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:52.666133 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9efaa1-c3aa-4699-be7c-cb2a1f129d79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:52.667499 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9efaa1-c3aa-4699-be7c-cb2a1f129d79 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:52.678348 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 547/1087] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:52.679228 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9efaa1-c3aa-4699-be7c-cb2a1f129d79 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:52.705294 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3137622c-53a4-4ef2-a89d-ec2bb7c28c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:52.706020 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3137622c-53a4-4ef2-a89d-ec2bb7c28c1c None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:52.706318 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3137622c-53a4-4ef2-a89d-ec2bb7c28c1c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:52.706590 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3137622c-53a4-4ef2-a89d-ec2bb7c28c1c None admin] PATH_INFO: `/v3/projects/e1e943c8e9924e0bab7d96b25ee1fd5b/users/4cc5e55c56fa46f4b5432867d44a7005/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:52.709290 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3137622c-53a4-4ef2-a89d-ec2bb7c28c1c None admin] RBAC: Authorizing `identity:create_grant(project_id=e1e943c8e9924e0bab7d96b25ee1fd5b, user_id=4cc5e55c56fa46f4b5432867d44a7005, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:52.710770 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3137622c-53a4-4ef2-a89d-ec2bb7c28c1c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:52.728624 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 541/1088] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 15:29:52 2022] PUT /identity/v3/projects/e1e943c8e9924e0bab7d96b25ee1fd5b/users/4cc5e55c56fa46f4b5432867d44a7005/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:29:52.746869 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:52.804641 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9efaa1-c3aa-4699-be7c-cb2a1f129d79 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:52.844281 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c17a97-707e-42ee-8c41-3cdab5713eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:52.845251 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c17a97-707e-42ee-8c41-3cdab5713eb5 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:52.845753 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c17a97-707e-42ee-8c41-3cdab5713eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:52.846229 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c17a97-707e-42ee-8c41-3cdab5713eb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:52.859608 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c17a97-707e-42ee-8c41-3cdab5713eb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:52.861383 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c17a97-707e-42ee-8c41-3cdab5713eb5 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:52.877436 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-a2c17a97-707e-42ee-8c41-3cdab5713eb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:52.877436 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:52.877436 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:52.877436 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:52.877436 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:52.877436 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:52.877436 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:52.877436 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:52.877436 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:52.877436 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:52.877436 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:52.877436 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:52.877436 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:52.877436 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:52.877436 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:52.877436 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:52.878566 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:52.878566 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:52.878566 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:52.878566 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:52.878566 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:52.878566 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:52.878566 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:52.878566 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:52.878566 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:52.878566 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:52.878566 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:52.878566 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:52.878566 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:52.878566 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:52.878566 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:52.878566 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:52.878566 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:52.880285 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:52.880285 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:52.880285 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:52.880285 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 548/1089] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:52.886746 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3137622c-53a4-4ef2-a89d-ec2bb7c28c1c None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:52.979259 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:53.000586 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3137622c-53a4-4ef2-a89d-ec2bb7c28c1c None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:53.041726 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c66e74-1f63-4f7c-b49f-3f526c185454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:53.042905 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c66e74-1f63-4f7c-b49f-3f526c185454 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:53.043177 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c66e74-1f63-4f7c-b49f-3f526c185454 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:53.043452 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c66e74-1f63-4f7c-b49f-3f526c185454 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:53.044096 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c66e74-1f63-4f7c-b49f-3f526c185454 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:53.044394 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c66e74-1f63-4f7c-b49f-3f526c185454 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:53.045750 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c66e74-1f63-4f7c-b49f-3f526c185454 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:53.058638 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 542/1090] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:53.064267 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c17a97-707e-42ee-8c41-3cdab5713eb5 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:53.164102 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c17a97-707e-42ee-8c41-3cdab5713eb5 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:53.166666 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:53.198009 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008985aa-31fc-4ddb-9097-dfaedf42953f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:53.198633 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008985aa-31fc-4ddb-9097-dfaedf42953f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:53.198907 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008985aa-31fc-4ddb-9097-dfaedf42953f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:53.199171 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008985aa-31fc-4ddb-9097-dfaedf42953f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:53.206516 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008985aa-31fc-4ddb-9097-dfaedf42953f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:53.208582 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008985aa-31fc-4ddb-9097-dfaedf42953f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:53.219433 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-008985aa-31fc-4ddb-9097-dfaedf42953f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 28 15:29:53.219433 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:53.219433 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:53.219433 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:53.219433 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:53.219433 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:53.219433 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:53.219433 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:53.219433 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:53.219433 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:53.219433 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:53.219433 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:53.219433 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:29:53.219433 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:29:53.219433 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:29:53.219433 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:29:53.220558 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:53.220558 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:53.220558 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:29:53.220558 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:53.220558 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:29:53.220558 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:29:53.220558 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:53.220558 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:53.220558 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:53.220558 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:53.220558 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:53.220558 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:53.220558 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:53.220558 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:53.220558 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:53.220558 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:53.220558 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:53.220558 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:53.221514 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:53.221514 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:53.221514 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:29:53.221514 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:29:53.221514 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:29:53.221514 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:29:53.221514 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:29:53.221514 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:29:53.221514 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 28 15:29:53.221514 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:53.221514 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 549/1091] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Sep 28 15:29:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:53.226291 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c66e74-1f63-4f7c-b49f-3f526c185454 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:53.325573 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c66e74-1f63-4f7c-b49f-3f526c185454 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:53.357668 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130d417a-de21-4877-9c4c-e09280416930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:53.358283 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d417a-de21-4877-9c4c-e09280416930 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:53.358560 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d417a-de21-4877-9c4c-e09280416930 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:53.358824 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d417a-de21-4877-9c4c-e09280416930 None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:53.359526 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130d417a-de21-4877-9c4c-e09280416930 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:53.359825 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130d417a-de21-4877-9c4c-e09280416930 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:53.361218 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130d417a-de21-4877-9c4c-e09280416930 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:53.370158 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:53.379062 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 543/1092] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Sep 28 15:29:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:53.385593 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008985aa-31fc-4ddb-9097-dfaedf42953f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:53.480636 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008985aa-31fc-4ddb-9097-dfaedf42953f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:53.514744 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e398f271-5f4a-4cd8-8095-fc097db8c63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:53.515346 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e398f271-5f4a-4cd8-8095-fc097db8c63b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:53.515623 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e398f271-5f4a-4cd8-8095-fc097db8c63b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:53.515895 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e398f271-5f4a-4cd8-8095-fc097db8c63b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:53.545923 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:53.582652 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e398f271-5f4a-4cd8-8095-fc097db8c63b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:53.584084 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e398f271-5f4a-4cd8-8095-fc097db8c63b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:53.685080 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 550/1093] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:29:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 28 15:29:53.686347 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130d417a-de21-4877-9c4c-e09280416930 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:53.782048 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130d417a-de21-4877-9c4c-e09280416930 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:53.815330 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80feb4b8-d942-4c6d-9c23-eaac2c4bd88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:53.815954 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80feb4b8-d942-4c6d-9c23-eaac2c4bd88b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:53.816229 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80feb4b8-d942-4c6d-9c23-eaac2c4bd88b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:53.816498 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80feb4b8-d942-4c6d-9c23-eaac2c4bd88b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:53.827012 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80feb4b8-d942-4c6d-9c23-eaac2c4bd88b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:53.828546 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80feb4b8-d942-4c6d-9c23-eaac2c4bd88b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:53.847036 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-80feb4b8-d942-4c6d-9c23-eaac2c4bd88b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:53.847036 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:53.847036 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:53.847036 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:53.847036 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:53.847036 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:53.847036 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:53.847036 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:53.847036 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:53.847036 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:53.847036 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:53.847036 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:53.847036 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:29:53.847036 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:29:53.847036 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:29:53.847036 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:29:53.848332 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:53.848332 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:53.848332 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:53.848332 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:53.848332 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:53.848332 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:53.848332 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:53.848332 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:53.848332 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:53.848332 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:53.848332 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:53.848332 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:53.848332 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:53.848332 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:53.848332 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:53.848332 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:53.848332 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:29:53.849242 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:29:53.849242 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:29:53.849242 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:53.849242 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 544/1094] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:53.853877 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e398f271-5f4a-4cd8-8095-fc097db8c63b None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:53.913160 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:53.949496 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e398f271-5f4a-4cd8-8095-fc097db8c63b None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:53.983356 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f5b2d-6a72-4881-95e0-c23ac5120f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:53.983959 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f5b2d-6a72-4881-95e0-c23ac5120f2a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:53.984237 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f5b2d-6a72-4881-95e0-c23ac5120f2a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:53.984506 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f5b2d-6a72-4881-95e0-c23ac5120f2a None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:53.985130 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484f5b2d-6a72-4881-95e0-c23ac5120f2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:53.985428 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484f5b2d-6a72-4881-95e0-c23ac5120f2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:53.986816 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484f5b2d-6a72-4881-95e0-c23ac5120f2a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:53.998298 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 551/1095] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:29:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:54.003825 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80feb4b8-d942-4c6d-9c23-eaac2c4bd88b None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:54.054499 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:54.104171 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80feb4b8-d942-4c6d-9c23-eaac2c4bd88b None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:54.141377 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38491967-a2e4-4ba0-a730-3b1fe803f4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:54.142034 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38491967-a2e4-4ba0-a730-3b1fe803f4e2 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:54.142306 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38491967-a2e4-4ba0-a730-3b1fe803f4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:54.142569 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38491967-a2e4-4ba0-a730-3b1fe803f4e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:54.148963 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38491967-a2e4-4ba0-a730-3b1fe803f4e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:54.150413 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38491967-a2e4-4ba0-a730-3b1fe803f4e2 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:54.161719 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-38491967-a2e4-4ba0-a730-3b1fe803f4e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:54.161719 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:54.161719 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:54.161719 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:54.161719 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:54.161719 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:54.161719 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:54.161719 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:54.161719 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:54.161719 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:54.161719 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:54.161719 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:54.161719 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:29:54.161719 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:29:54.161719 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:29:54.162795 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:29:54.162795 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:54.162795 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:54.162795 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:54.162795 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:54.162795 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:54.162795 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:54.162795 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:54.162795 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:54.162795 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:54.162795 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:54.162795 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:54.162795 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:54.162795 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:54.162795 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:54.162795 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:54.162795 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:54.163696 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:29:54.163696 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:29:54.163696 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:29:54.163696 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:29:54.163696 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:29:54.163696 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:29:54.163696 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:29:54.163696 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:29:54.163696 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 545/1096] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:29:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:54.169076 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f5b2d-6a72-4881-95e0-c23ac5120f2a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:54.264313 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f5b2d-6a72-4881-95e0-c23ac5120f2a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:54.286102 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:54.297423 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867d38c-7f95-456c-9c79-8020ed769d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:54.298057 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867d38c-7f95-456c-9c79-8020ed769d4a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:54.298337 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867d38c-7f95-456c-9c79-8020ed769d4a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:54.298606 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867d38c-7f95-456c-9c79-8020ed769d4a None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:54.299281 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3867d38c-7f95-456c-9c79-8020ed769d4a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:54.299583 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3867d38c-7f95-456c-9c79-8020ed769d4a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:54.301005 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3867d38c-7f95-456c-9c79-8020ed769d4a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:54.313446 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 552/1097] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 15:29:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:54.319939 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38491967-a2e4-4ba0-a730-3b1fe803f4e2 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:54.415693 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38491967-a2e4-4ba0-a730-3b1fe803f4e2 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:54.425439 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:54.448720 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e9ceaa-f332-4f11-a1c4-144a7fb7c4c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:54.449319 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e9ceaa-f332-4f11-a1c4-144a7fb7c4c3 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:54.449593 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e9ceaa-f332-4f11-a1c4-144a7fb7c4c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:54.449884 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e9ceaa-f332-4f11-a1c4-144a7fb7c4c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:54.450597 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e9ceaa-f332-4f11-a1c4-144a7fb7c4c3 None admin] RBAC: Adding query filter params (role.id=769ffe2b49154917b28e916692c53d5f, scope.project.id=3dc6c5f5fcd34922b0d09cd2aaed6cf1, user.id=8daf14b64bf04b2ba74d2fe1fd41ff71) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:54.450904 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e9ceaa-f332-4f11-a1c4-144a7fb7c4c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:54.452316 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e9ceaa-f332-4f11-a1c4-144a7fb7c4c3 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:54.458024 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 546/1098] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 15:29:54 2022] GET /identity/v3/role_assignments?user.id=8daf14b64bf04b2ba74d2fe1fd41ff71&scope.project.id=3dc6c5f5fcd34922b0d09cd2aaed6cf1&role.id=769ffe2b49154917b28e916692c53d5f => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:29:54.919204 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929b167-9b3d-45f1-b352-f83ab2aeb77d None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:54.919719 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929b167-9b3d-45f1-b352-f83ab2aeb77d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:54.919831 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929b167-9b3d-45f1-b352-f83ab2aeb77d None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:54.921083 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 553/1099] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:54.926572 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4671d25-4a09-42de-b11d-226565b4693d None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:54.927005 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4671d25-4a09-42de-b11d-226565b4693d None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:54.927126 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4671d25-4a09-42de-b11d-226565b4693d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:54.945129 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-c4671d25-4a09-42de-b11d-226565b4693d None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:54.946347 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c4671d25-4a09-42de-b11d-226565b4693d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:55.014807 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 547/1100] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:29:55.277677 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5929b167-9b3d-45f1-b352-f83ab2aeb77d None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:55.375857 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5929b167-9b3d-45f1-b352-f83ab2aeb77d None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:55.408922 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67023f6a-1c6e-49ac-95ad-52c08dad07aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:55.409532 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67023f6a-1c6e-49ac-95ad-52c08dad07aa None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:55.409810 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67023f6a-1c6e-49ac-95ad-52c08dad07aa None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:55.410109 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67023f6a-1c6e-49ac-95ad-52c08dad07aa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:55.419720 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67023f6a-1c6e-49ac-95ad-52c08dad07aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:29:55.424441 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-67023f6a-1c6e-49ac-95ad-52c08dad07aa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:29:55.424441 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:29:55.424441 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:29:55.424441 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:29:55.424441 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:29:55.424441 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:29:55.424441 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:29:55.424441 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:29:55.424441 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:29:55.424441 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:29:55.424441 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:29:55.424441 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:29:55.424441 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:29:55.424441 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:29:55.424441 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:29:55.424441 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:29:55.425522 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:29:55.425522 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:29:55.425522 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:29:55.425522 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:29:55.425522 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:29:55.425522 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:29:55.425522 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:29:55.425522 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:29:55.425522 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:29:55.425522 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:29:55.425522 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:29:55.425522 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:29:55.425522 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:29:55.425522 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:29:55.425522 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:29:55.425522 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:29:55.426454 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:29:55.426454 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:29:55.426454 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:29:55.426454 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:29:55.426454 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:29:55.426454 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:29:55.426454 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:29:55.426454 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:29:55.426454 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:29:55.426454 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:29:55.426454 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:29:55.426454 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:29:55.426454 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 554/1101] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 15:29:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:29:55.431596 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4671d25-4a09-42de-b11d-226565b4693d None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:55.526062 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4671d25-4a09-42de-b11d-226565b4693d None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:55.528287 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:55.558573 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476272c3-4bb5-489b-bb0e-fe5d0e16e00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:55.559174 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476272c3-4bb5-489b-bb0e-fe5d0e16e00e None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:55.559448 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476272c3-4bb5-489b-bb0e-fe5d0e16e00e None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:55.559719 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476272c3-4bb5-489b-bb0e-fe5d0e16e00e None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:55.560364 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476272c3-4bb5-489b-bb0e-fe5d0e16e00e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:55.560662 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476272c3-4bb5-489b-bb0e-fe5d0e16e00e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:55.562025 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476272c3-4bb5-489b-bb0e-fe5d0e16e00e None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:55.570414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 548/1102] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:29:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:55.576157 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67023f6a-1c6e-49ac-95ad-52c08dad07aa None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:55.621025 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:55.672795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67023f6a-1c6e-49ac-95ad-52c08dad07aa None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:55.705051 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36059852-e909-4a04-afbe-354e715e58d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:55.705652 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36059852-e909-4a04-afbe-354e715e58d5 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:55.705954 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36059852-e909-4a04-afbe-354e715e58d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:55.706225 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36059852-e909-4a04-afbe-354e715e58d5 None admin] PATH_INFO: `/v3/users/4cc5e55c56fa46f4b5432867d44a7005` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:55.707306 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36059852-e909-4a04-afbe-354e715e58d5 None admin] RBAC: Authorizing `identity:get_user(user_id=4cc5e55c56fa46f4b5432867d44a7005)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:55.708716 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36059852-e909-4a04-afbe-354e715e58d5 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:55.710276 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 555/1103] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:29:55 2022] GET /identity/v3/users/4cc5e55c56fa46f4b5432867d44a7005 => generated 285 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:55.715903 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476272c3-4bb5-489b-bb0e-fe5d0e16e00e None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:55.811755 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476272c3-4bb5-489b-bb0e-fe5d0e16e00e None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:55.843699 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3519b1d3-ec35-4334-adbd-f435441efe44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:55.844288 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3519b1d3-ec35-4334-adbd-f435441efe44 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:55.844561 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3519b1d3-ec35-4334-adbd-f435441efe44 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:55.844828 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3519b1d3-ec35-4334-adbd-f435441efe44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:55.876599 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:55.909994 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3519b1d3-ec35-4334-adbd-f435441efe44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:55.911934 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3519b1d3-ec35-4334-adbd-f435441efe44 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:56.006106 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 549/1104] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:29:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 15:29:56.011995 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36059852-e909-4a04-afbe-354e715e58d5 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:56.107410 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36059852-e909-4a04-afbe-354e715e58d5 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:56.120240 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:56.140424 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725486f4-6de0-4997-ae6b-949738cb5bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:56.141040 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725486f4-6de0-4997-ae6b-949738cb5bba None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:56.141319 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725486f4-6de0-4997-ae6b-949738cb5bba None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:56.141590 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725486f4-6de0-4997-ae6b-949738cb5bba None admin] PATH_INFO: `/v3/projects/e1e943c8e9924e0bab7d96b25ee1fd5b` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:56.142708 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725486f4-6de0-4997-ae6b-949738cb5bba None admin] RBAC: Authorizing `identity:get_project(project_id=e1e943c8e9924e0bab7d96b25ee1fd5b)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:56.144127 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725486f4-6de0-4997-ae6b-949738cb5bba None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:56.145757 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 556/1105] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:29:56 2022] GET /identity/v3/projects/e1e943c8e9924e0bab7d96b25ee1fd5b => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:56.151811 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3519b1d3-ec35-4334-adbd-f435441efe44 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:56.218234 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:56.253138 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3519b1d3-ec35-4334-adbd-f435441efe44 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:56.286639 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f204b999-bce9-4101-8256-f0be88904c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:56.287231 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204b999-bce9-4101-8256-f0be88904c73 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:56.287504 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204b999-bce9-4101-8256-f0be88904c73 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:56.287770 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204b999-bce9-4101-8256-f0be88904c73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:56.288482 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f204b999-bce9-4101-8256-f0be88904c73 None admin] RBAC: Adding query filter params (role.id=62d96ac9513c452894fc67b442cfc59f, scope.project.id=e1e943c8e9924e0bab7d96b25ee1fd5b, user.id=4cc5e55c56fa46f4b5432867d44a7005) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:29:56.288785 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f204b999-bce9-4101-8256-f0be88904c73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:56.290255 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f204b999-bce9-4101-8256-f0be88904c73 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:56.296114 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 550/1106] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 15:29:56 2022] GET /identity/v3/role_assignments?user.id=4cc5e55c56fa46f4b5432867d44a7005&scope.project.id=e1e943c8e9924e0bab7d96b25ee1fd5b&role.id=62d96ac9513c452894fc67b442cfc59f => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:58.449450 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea59ad-5747-405e-aa53-f8e924091cb3 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:58.450029 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea59ad-5747-405e-aa53-f8e924091cb3 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:58.450126 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea59ad-5747-405e-aa53-f8e924091cb3 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:58.451485 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 557/1107] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:29:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:29:58.456979 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12579c8b-10fc-4069-87f0-566a3023bf02 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:58.457377 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12579c8b-10fc-4069-87f0-566a3023bf02 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:58.457620 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12579c8b-10fc-4069-87f0-566a3023bf02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:58.481303 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-12579c8b-10fc-4069-87f0-566a3023bf02 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:29:58.482548 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-12579c8b-10fc-4069-87f0-566a3023bf02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:29:58.564716 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 551/1108] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:29:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:29:58.840475 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea59ad-5747-405e-aa53-f8e924091cb3 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:58.937580 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea59ad-5747-405e-aa53-f8e924091cb3 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:58.970896 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b1aef5-ce89-495f-bea1-4774ee43996a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:58.971524 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1aef5-ce89-495f-bea1-4774ee43996a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:58.971783 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1aef5-ce89-495f-bea1-4774ee43996a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:58.972052 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1aef5-ce89-495f-bea1-4774ee43996a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:58.973162 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b1aef5-ce89-495f-bea1-4774ee43996a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:58.974685 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b1aef5-ce89-495f-bea1-4774ee43996a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:58.976413 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 558/1109] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:29:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:29:58.982565 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12579c8b-10fc-4069-87f0-566a3023bf02 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:29:59.039258 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:29:59.077506 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12579c8b-10fc-4069-87f0-566a3023bf02 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:29:59.109746 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceabe5c3-3127-445a-901d-3cf18c9e942a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:29:59.110421 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceabe5c3-3127-445a-901d-3cf18c9e942a None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:29:59.110697 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceabe5c3-3127-445a-901d-3cf18c9e942a None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:29:59.110962 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceabe5c3-3127-445a-901d-3cf18c9e942a None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:29:59.111593 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceabe5c3-3127-445a-901d-3cf18c9e942a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:29:59.112997 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceabe5c3-3127-445a-901d-3cf18c9e942a None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:29:59.133431 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 552/1110] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 15:29:58 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:30:01.290913 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986f85c-d6a7-4ce4-b941-e6bcf1094ee6 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:01.291436 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986f85c-d6a7-4ce4-b941-e6bcf1094ee6 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:01.291436 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986f85c-d6a7-4ce4-b941-e6bcf1094ee6 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:01.292784 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 559/1111] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:30:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:01.298033 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df155d6-ce87-4256-b642-99f199127360 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:01.298434 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df155d6-ce87-4256-b642-99f199127360 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:01.298586 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df155d6-ce87-4256-b642-99f199127360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:01.317119 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-2df155d6-ce87-4256-b642-99f199127360 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:01.318338 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-2df155d6-ce87-4256-b642-99f199127360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:01.383021 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 553/1112] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:30:01.649530 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d986f85c-d6a7-4ce4-b941-e6bcf1094ee6 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:01.697519 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:01.746776 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d986f85c-d6a7-4ce4-b941-e6bcf1094ee6 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:01.779129 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a594b31c-df95-4140-8d1c-379170cb4330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:01.779736 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594b31c-df95-4140-8d1c-379170cb4330 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:01.780014 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594b31c-df95-4140-8d1c-379170cb4330 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:01.780280 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594b31c-df95-4140-8d1c-379170cb4330 None admin] PATH_INFO: `/v3/roles/5038c8140b7145ceb7494210a5c9f9f0` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:01.781627 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a594b31c-df95-4140-8d1c-379170cb4330 None admin] RBAC: Authorizing `identity:get_role(role_id=5038c8140b7145ceb7494210a5c9f9f0)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:01.783015 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a594b31c-df95-4140-8d1c-379170cb4330 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:01.784295 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 560/1113] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:30:01 2022] GET /identity/v3/roles/5038c8140b7145ceb7494210a5c9f9f0 => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:01.790695 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df155d6-ce87-4256-b642-99f199127360 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:01.833761 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:01.884753 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df155d6-ce87-4256-b642-99f199127360 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:01.916675 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab3be60-747b-4aea-85b2-0e24dbd359ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:01.917225 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3be60-747b-4aea-85b2-0e24dbd359ed None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:01.917498 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3be60-747b-4aea-85b2-0e24dbd359ed None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:01.917768 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3be60-747b-4aea-85b2-0e24dbd359ed None admin] PATH_INFO: `/v3/users/a50e775f8e4f4f6ba8b83662fd84f12c` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:01.930713 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab3be60-747b-4aea-85b2-0e24dbd359ed None admin] RBAC: Authorizing `identity:get_user(user_id=a50e775f8e4f4f6ba8b83662fd84f12c)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:01.932131 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab3be60-747b-4aea-85b2-0e24dbd359ed None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:01.933667 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 554/1114] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:30:01 2022] GET /identity/v3/users/a50e775f8e4f4f6ba8b83662fd84f12c => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:01.939425 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a594b31c-df95-4140-8d1c-379170cb4330 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:02.037394 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a594b31c-df95-4140-8d1c-379170cb4330 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:02.053007 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:02.070578 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4055fe7b-609a-46ad-bcc3-c79039a9b840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:02.071176 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4055fe7b-609a-46ad-bcc3-c79039a9b840 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:02.071457 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4055fe7b-609a-46ad-bcc3-c79039a9b840 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:02.071728 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4055fe7b-609a-46ad-bcc3-c79039a9b840 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:02.135771 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4055fe7b-609a-46ad-bcc3-c79039a9b840 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:02.137228 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4055fe7b-609a-46ad-bcc3-c79039a9b840 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:02.228684 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 561/1115] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:30:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 15:30:02.234515 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab3be60-747b-4aea-85b2-0e24dbd359ed None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:02.340820 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab3be60-747b-4aea-85b2-0e24dbd359ed None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:02.343057 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:02.374829 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88aca87-77fe-4095-8bc1-4fe8ecebb314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:02.375446 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88aca87-77fe-4095-8bc1-4fe8ecebb314 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:02.375722 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88aca87-77fe-4095-8bc1-4fe8ecebb314 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:02.375991 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88aca87-77fe-4095-8bc1-4fe8ecebb314 None admin] PATH_INFO: `/v3/projects/e1e943c8e9924e0bab7d96b25ee1fd5b` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:02.377086 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88aca87-77fe-4095-8bc1-4fe8ecebb314 None admin] RBAC: Authorizing `identity:get_project(project_id=e1e943c8e9924e0bab7d96b25ee1fd5b)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:02.378540 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88aca87-77fe-4095-8bc1-4fe8ecebb314 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:02.380835 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 555/1116] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:30:02 2022] GET /identity/v3/projects/e1e943c8e9924e0bab7d96b25ee1fd5b => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:02.387076 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4055fe7b-609a-46ad-bcc3-c79039a9b840 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:02.403968 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:02.485334 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4055fe7b-609a-46ad-bcc3-c79039a9b840 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:02.518449 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dab7b5-f59b-409f-a156-cae382f56c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:02.519042 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dab7b5-f59b-409f-a156-cae382f56c1b None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:02.519425 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dab7b5-f59b-409f-a156-cae382f56c1b None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:02.519704 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dab7b5-f59b-409f-a156-cae382f56c1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:02.520417 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dab7b5-f59b-409f-a156-cae382f56c1b None admin] RBAC: Adding query filter params (role.id=5038c8140b7145ceb7494210a5c9f9f0, scope.project.id=e1e943c8e9924e0bab7d96b25ee1fd5b, user.id=a50e775f8e4f4f6ba8b83662fd84f12c) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:02.520723 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dab7b5-f59b-409f-a156-cae382f56c1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:02.522164 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dab7b5-f59b-409f-a156-cae382f56c1b None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:02.527285 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 562/1117] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 15:30:02 2022] GET /identity/v3/role_assignments?user.id=a50e775f8e4f4f6ba8b83662fd84f12c&scope.project.id=e1e943c8e9924e0bab7d96b25ee1fd5b&role.id=5038c8140b7145ceb7494210a5c9f9f0 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:04.628278 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f51d2d-42b2-4dfa-af4a-567aa7715757 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:04.628849 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f51d2d-42b2-4dfa-af4a-567aa7715757 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:04.628849 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f51d2d-42b2-4dfa-af4a-567aa7715757 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:04.630425 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 556/1118] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:30:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:04.635693 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b030abb-e351-4c56-86b5-a3e0edfeb54b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:04.636128 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b030abb-e351-4c56-86b5-a3e0edfeb54b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:04.636235 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b030abb-e351-4c56-86b5-a3e0edfeb54b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:04.654395 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-2b030abb-e351-4c56-86b5-a3e0edfeb54b None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:04.655562 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2b030abb-e351-4c56-86b5-a3e0edfeb54b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:04.718367 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 563/1119] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:30:04.987377 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f51d2d-42b2-4dfa-af4a-567aa7715757 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:05.084542 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f51d2d-42b2-4dfa-af4a-567aa7715757 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:05.116045 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e9a2a-8107-437e-89fa-e9d6117f8854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:05.116656 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e9a2a-8107-437e-89fa-e9d6117f8854 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:05.116932 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e9a2a-8107-437e-89fa-e9d6117f8854 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:05.117202 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e9a2a-8107-437e-89fa-e9d6117f8854 None admin] PATH_INFO: `/v3/roles/5038c8140b7145ceb7494210a5c9f9f0` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:05.118583 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3e9a2a-8107-437e-89fa-e9d6117f8854 None admin] RBAC: Authorizing `identity:get_role(role_id=5038c8140b7145ceb7494210a5c9f9f0)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:05.119921 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3e9a2a-8107-437e-89fa-e9d6117f8854 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:05.121185 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 557/1120] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:30:04 2022] GET /identity/v3/roles/5038c8140b7145ceb7494210a5c9f9f0 => generated 223 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:05.126793 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b030abb-e351-4c56-86b5-a3e0edfeb54b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:05.138800 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:05.234175 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b030abb-e351-4c56-86b5-a3e0edfeb54b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:05.268969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad1b83f-4ce3-43e0-ab74-0275fa690a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:05.269563 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad1b83f-4ce3-43e0-ab74-0275fa690a9f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:05.269838 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad1b83f-4ce3-43e0-ab74-0275fa690a9f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:05.270131 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad1b83f-4ce3-43e0-ab74-0275fa690a9f None admin] PATH_INFO: `/v3/users/a50e775f8e4f4f6ba8b83662fd84f12c` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:05.271236 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad1b83f-4ce3-43e0-ab74-0275fa690a9f None admin] RBAC: Authorizing `identity:get_user(user_id=a50e775f8e4f4f6ba8b83662fd84f12c)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:05.272674 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad1b83f-4ce3-43e0-ab74-0275fa690a9f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:05.274255 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 564/1121] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:30:05 2022] GET /identity/v3/users/a50e775f8e4f4f6ba8b83662fd84f12c => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:05.280031 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e9a2a-8107-437e-89fa-e9d6117f8854 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:05.322079 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:05.378334 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e9a2a-8107-437e-89fa-e9d6117f8854 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:05.412594 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317fd8c7-4966-40ec-b426-85f6cd54134e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:05.413206 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317fd8c7-4966-40ec-b426-85f6cd54134e None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:05.413486 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317fd8c7-4966-40ec-b426-85f6cd54134e None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:05.413758 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317fd8c7-4966-40ec-b426-85f6cd54134e None admin] PATH_INFO: `/v3/projects/e1e943c8e9924e0bab7d96b25ee1fd5b` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:05.414889 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317fd8c7-4966-40ec-b426-85f6cd54134e None admin] RBAC: Authorizing `identity:get_project(project_id=e1e943c8e9924e0bab7d96b25ee1fd5b)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:05.416303 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317fd8c7-4966-40ec-b426-85f6cd54134e None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:05.417923 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 558/1122] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:30:05 2022] GET /identity/v3/projects/e1e943c8e9924e0bab7d96b25ee1fd5b => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:05.424100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad1b83f-4ce3-43e0-ab74-0275fa690a9f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:05.497516 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:05.522206 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad1b83f-4ce3-43e0-ab74-0275fa690a9f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:05.555058 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc77b0-9b21-4cde-afdc-42f80daa08e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:05.555702 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc77b0-9b21-4cde-afdc-42f80daa08e5 None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:05.555982 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc77b0-9b21-4cde-afdc-42f80daa08e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:05.556254 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc77b0-9b21-4cde-afdc-42f80daa08e5 None admin] PATH_INFO: `/v3/projects/e1e943c8e9924e0bab7d96b25ee1fd5b/users/a50e775f8e4f4f6ba8b83662fd84f12c/roles/5038c8140b7145ceb7494210a5c9f9f0` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:05.558373 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edc77b0-9b21-4cde-afdc-42f80daa08e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=e1e943c8e9924e0bab7d96b25ee1fd5b, user_id=a50e775f8e4f4f6ba8b83662fd84f12c, role_id=5038c8140b7145ceb7494210a5c9f9f0)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:05.559801 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edc77b0-9b21-4cde-afdc-42f80daa08e5 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:05.572484 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 565/1123] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 15:30:05 2022] PUT /identity/v3/projects/e1e943c8e9924e0bab7d96b25ee1fd5b/users/a50e775f8e4f4f6ba8b83662fd84f12c/roles/5038c8140b7145ceb7494210a5c9f9f0 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:30:07.656871 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffed62d-1547-43f5-a0ec-920f80e26432 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:07.657405 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffed62d-1547-43f5-a0ec-920f80e26432 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:07.657405 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffed62d-1547-43f5-a0ec-920f80e26432 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:07.658759 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 559/1124] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:30:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:07.663857 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68794ecc-0fc8-4800-a0e4-62ce2f84855f None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:07.664273 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68794ecc-0fc8-4800-a0e4-62ce2f84855f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:07.664398 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68794ecc-0fc8-4800-a0e4-62ce2f84855f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:07.682673 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-68794ecc-0fc8-4800-a0e4-62ce2f84855f None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:07.683833 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-68794ecc-0fc8-4800-a0e4-62ce2f84855f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:07.746460 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 566/1125] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:30:08.014130 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffed62d-1547-43f5-a0ec-920f80e26432 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:08.110773 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffed62d-1547-43f5-a0ec-920f80e26432 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:08.120065 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:08.144607 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e738f409-c2ad-4387-b745-b9317f1e6a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:08.145220 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738f409-c2ad-4387-b745-b9317f1e6a43 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:08.145495 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738f409-c2ad-4387-b745-b9317f1e6a43 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:08.145763 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738f409-c2ad-4387-b745-b9317f1e6a43 None admin] PATH_INFO: `/v3/roles/5038c8140b7145ceb7494210a5c9f9f0` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:08.147142 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e738f409-c2ad-4387-b745-b9317f1e6a43 None admin] RBAC: Authorizing `identity:get_role(role_id=5038c8140b7145ceb7494210a5c9f9f0)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:08.148485 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e738f409-c2ad-4387-b745-b9317f1e6a43 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:08.149767 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 560/1126] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:30:08 2022] GET /identity/v3/roles/5038c8140b7145ceb7494210a5c9f9f0 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:08.155486 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68794ecc-0fc8-4800-a0e4-62ce2f84855f None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:08.217534 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:08.251240 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68794ecc-0fc8-4800-a0e4-62ce2f84855f None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:08.284169 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee84bbf-f0ec-4e9c-b81d-1d0a4c3efbdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:08.285383 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee84bbf-f0ec-4e9c-b81d-1d0a4c3efbdf None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:08.285662 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee84bbf-f0ec-4e9c-b81d-1d0a4c3efbdf None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:08.285965 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee84bbf-f0ec-4e9c-b81d-1d0a4c3efbdf None admin] PATH_INFO: `/v3/users/a50e775f8e4f4f6ba8b83662fd84f12c` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:08.287163 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee84bbf-f0ec-4e9c-b81d-1d0a4c3efbdf None admin] RBAC: Authorizing `identity:get_user(user_id=a50e775f8e4f4f6ba8b83662fd84f12c)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:08.289551 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee84bbf-f0ec-4e9c-b81d-1d0a4c3efbdf None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:08.292141 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 567/1127] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:30:08 2022] GET /identity/v3/users/a50e775f8e4f4f6ba8b83662fd84f12c => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:08.297892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e738f409-c2ad-4387-b745-b9317f1e6a43 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:08.393103 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e738f409-c2ad-4387-b745-b9317f1e6a43 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:08.425640 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42af85f3-132c-4f42-b2be-f822a513a4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:08.426257 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af85f3-132c-4f42-b2be-f822a513a4e4 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:08.426540 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af85f3-132c-4f42-b2be-f822a513a4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:08.426809 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af85f3-132c-4f42-b2be-f822a513a4e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:08.469839 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:08.492730 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42af85f3-132c-4f42-b2be-f822a513a4e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:08.494256 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42af85f3-132c-4f42-b2be-f822a513a4e4 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:08.589803 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 561/1128] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:30:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 15:30:08.595940 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee84bbf-f0ec-4e9c-b81d-1d0a4c3efbdf None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:08.691568 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee84bbf-f0ec-4e9c-b81d-1d0a4c3efbdf None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:08.724214 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e93d3a-2b2c-4b09-be6b-3f1de7cbe377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:08.724824 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e93d3a-2b2c-4b09-be6b-3f1de7cbe377 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:08.725102 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e93d3a-2b2c-4b09-be6b-3f1de7cbe377 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:08.725373 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e93d3a-2b2c-4b09-be6b-3f1de7cbe377 None admin] PATH_INFO: `/v3/projects/e1e943c8e9924e0bab7d96b25ee1fd5b` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:08.726457 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e93d3a-2b2c-4b09-be6b-3f1de7cbe377 None admin] RBAC: Authorizing `identity:get_project(project_id=e1e943c8e9924e0bab7d96b25ee1fd5b)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:08.727876 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e93d3a-2b2c-4b09-be6b-3f1de7cbe377 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:08.729466 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 568/1129] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:30:08 2022] GET /identity/v3/projects/e1e943c8e9924e0bab7d96b25ee1fd5b => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:08.735533 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42af85f3-132c-4f42-b2be-f822a513a4e4 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:08.807983 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:08.830042 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42af85f3-132c-4f42-b2be-f822a513a4e4 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:08.862130 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe20204-32c5-4f5e-ab98-d52c5d118ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:08.862701 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe20204-32c5-4f5e-ab98-d52c5d118ad0 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:08.862974 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe20204-32c5-4f5e-ab98-d52c5d118ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:08.863237 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe20204-32c5-4f5e-ab98-d52c5d118ad0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:08.863935 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe20204-32c5-4f5e-ab98-d52c5d118ad0 None admin] RBAC: Adding query filter params (role.id=5038c8140b7145ceb7494210a5c9f9f0, scope.project.id=e1e943c8e9924e0bab7d96b25ee1fd5b, user.id=a50e775f8e4f4f6ba8b83662fd84f12c) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:08.864235 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe20204-32c5-4f5e-ab98-d52c5d118ad0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:08.865635 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe20204-32c5-4f5e-ab98-d52c5d118ad0 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:08.871096 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 562/1130] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 15:30:08 2022] GET /identity/v3/role_assignments?user.id=a50e775f8e4f4f6ba8b83662fd84f12c&scope.project.id=e1e943c8e9924e0bab7d96b25ee1fd5b&role.id=5038c8140b7145ceb7494210a5c9f9f0 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:11.012693 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51f16b-8714-4b7e-bbf9-6ab878dc24c4 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:11.013248 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51f16b-8714-4b7e-bbf9-6ab878dc24c4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:11.013248 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51f16b-8714-4b7e-bbf9-6ab878dc24c4 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:11.014596 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 569/1131] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:30:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:11.019901 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb876db7-7fb8-4d60-8eea-4da0fb8d2647 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:11.020299 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb876db7-7fb8-4d60-8eea-4da0fb8d2647 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:11.020431 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb876db7-7fb8-4d60-8eea-4da0fb8d2647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:11.038493 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-bb876db7-7fb8-4d60-8eea-4da0fb8d2647 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:11.039658 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-bb876db7-7fb8-4d60-8eea-4da0fb8d2647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:11.102784 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 563/1132] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:30:11.370080 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e51f16b-8714-4b7e-bbf9-6ab878dc24c4 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:11.373838 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:11.469226 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e51f16b-8714-4b7e-bbf9-6ab878dc24c4 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:11.504851 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e675152-d601-4afa-93d3-bc6714e3dd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:11.505483 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e675152-d601-4afa-93d3-bc6714e3dd3d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:11.505764 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e675152-d601-4afa-93d3-bc6714e3dd3d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:11.506063 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e675152-d601-4afa-93d3-bc6714e3dd3d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:11.507173 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e675152-d601-4afa-93d3-bc6714e3dd3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:11.508598 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e675152-d601-4afa-93d3-bc6714e3dd3d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:11.510199 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 570/1133] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:30:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:11.515964 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb876db7-7fb8-4d60-8eea-4da0fb8d2647 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:11.580886 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:11.616205 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb876db7-7fb8-4d60-8eea-4da0fb8d2647 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:11.654660 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ebb5c-6811-4cc9-b304-c88679598504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:11.655260 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ebb5c-6811-4cc9-b304-c88679598504 None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:11.655536 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ebb5c-6811-4cc9-b304-c88679598504 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:11.655805 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ebb5c-6811-4cc9-b304-c88679598504 None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:11.656452 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904ebb5c-6811-4cc9-b304-c88679598504 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:11.658312 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904ebb5c-6811-4cc9-b304-c88679598504 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:11.686859 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 564/1134] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Sep 28 15:30:11 2022] POST /identity/v3/projects => generated 311 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:30:13.841211 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf9f17-5c6d-43bb-a9c5-561321beda5b None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:13.841783 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf9f17-5c6d-43bb-a9c5-561321beda5b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:13.841783 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf9f17-5c6d-43bb-a9c5-561321beda5b None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:13.843113 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 571/1135] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:30:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:13.848958 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f5ab31-1bb5-4695-83f2-981f71eb2f5b None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:13.849378 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f5ab31-1bb5-4695-83f2-981f71eb2f5b None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:13.849493 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f5ab31-1bb5-4695-83f2-981f71eb2f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:13.867062 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-12f5ab31-1bb5-4695-83f2-981f71eb2f5b None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:13.868225 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-12f5ab31-1bb5-4695-83f2-981f71eb2f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:13.943165 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 565/1136] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:30:14.211701 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bf9f17-5c6d-43bb-a9c5-561321beda5b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:14.281706 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:14.311288 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bf9f17-5c6d-43bb-a9c5-561321beda5b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:14.343560 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfe01eb-afbd-4022-80e1-995794d150ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:14.344176 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe01eb-afbd-4022-80e1-995794d150ce None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:14.344450 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe01eb-afbd-4022-80e1-995794d150ce None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:14.344715 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe01eb-afbd-4022-80e1-995794d150ce None admin] PATH_INFO: `/v3/domains/default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:14.345785 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfe01eb-afbd-4022-80e1-995794d150ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:14.347305 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfe01eb-afbd-4022-80e1-995794d150ce None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:14.348828 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 572/1137] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:30:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:14.354696 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f5ab31-1bb5-4695-83f2-981f71eb2f5b None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:14.404686 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:14.450771 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f5ab31-1bb5-4695-83f2-981f71eb2f5b None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:14.483645 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85feaff4-6079-4fc6-b0d2-777d5b03d233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:14.484183 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85feaff4-6079-4fc6-b0d2-777d5b03d233 None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:14.484460 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85feaff4-6079-4fc6-b0d2-777d5b03d233 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:14.484727 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85feaff4-6079-4fc6-b0d2-777d5b03d233 None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:14.485385 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85feaff4-6079-4fc6-b0d2-777d5b03d233 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:14.486821 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85feaff4-6079-4fc6-b0d2-777d5b03d233 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:14.510594 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 566/1138] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 15:30:14 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:30:16.669073 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4ee82-b69c-4b0b-8b88-14de1388db6a None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:16.669632 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4ee82-b69c-4b0b-8b88-14de1388db6a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:16.669632 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4ee82-b69c-4b0b-8b88-14de1388db6a None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:16.670966 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 573/1139] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:30:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:16.676338 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6540f-916b-413f-85a1-39caa8e65158 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:16.676740 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6540f-916b-413f-85a1-39caa8e65158 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:16.676927 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6540f-916b-413f-85a1-39caa8e65158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:16.694508 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-daf6540f-916b-413f-85a1-39caa8e65158 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:16.695660 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-daf6540f-916b-413f-85a1-39caa8e65158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:16.758899 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 567/1140] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:30:17.030473 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b4ee82-b69c-4b0b-8b88-14de1388db6a None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:17.128236 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b4ee82-b69c-4b0b-8b88-14de1388db6a None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:17.160953 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a0638-6ffa-49ae-ad13-d59cbb0a98b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:17.161562 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a0638-6ffa-49ae-ad13-d59cbb0a98b5 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:17.161844 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a0638-6ffa-49ae-ad13-d59cbb0a98b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:17.162143 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a0638-6ffa-49ae-ad13-d59cbb0a98b5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:17.169418 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:17.172697 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040a0638-6ffa-49ae-ad13-d59cbb0a98b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:30:17.177993 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-040a0638-6ffa-49ae-ad13-d59cbb0a98b5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:30:17.177993 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:17.177993 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:17.177993 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:17.177993 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:17.177993 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:17.177993 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:17.177993 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:17.177993 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:17.177993 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:17.177993 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:17.177993 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:17.177993 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:30:17.177993 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:30:17.177993 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:30:17.177993 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:30:17.179105 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:30:17.179105 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:30:17.179105 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:30:17.179105 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:30:17.179105 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:17.179105 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:17.179105 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:17.179105 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:17.179105 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:17.179105 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:17.179105 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:17.179105 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:17.179105 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:17.179105 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:17.179105 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:17.179105 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:17.180006 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:17.180006 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:17.180006 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:17.180006 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:17.180006 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:30:17.180006 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:30:17.180006 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:30:17.180006 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:30:17.180006 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:30:17.180006 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:30:17.180006 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:30:17.180006 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:17.180006 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 574/1141] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 15:30:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:17.185348 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf6540f-916b-413f-85a1-39caa8e65158 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:17.223316 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:17.281341 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf6540f-916b-413f-85a1-39caa8e65158 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:17.313976 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87711dd9-bc85-4da2-8ba7-6334a82852eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:17.314571 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87711dd9-bc85-4da2-8ba7-6334a82852eb None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:17.314845 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87711dd9-bc85-4da2-8ba7-6334a82852eb None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:17.315201 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87711dd9-bc85-4da2-8ba7-6334a82852eb None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:17.316354 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87711dd9-bc85-4da2-8ba7-6334a82852eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:17.316648 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87711dd9-bc85-4da2-8ba7-6334a82852eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:17.318025 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87711dd9-bc85-4da2-8ba7-6334a82852eb None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:17.325702 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 568/1142] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:30:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:17.331715 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a0638-6ffa-49ae-ad13-d59cbb0a98b5 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:17.430855 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a0638-6ffa-49ae-ad13-d59cbb0a98b5 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:17.464139 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b13e49-d7fd-465e-982e-f808b2e68447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:17.464773 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b13e49-d7fd-465e-982e-f808b2e68447 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:17.465051 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b13e49-d7fd-465e-982e-f808b2e68447 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:17.465321 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b13e49-d7fd-465e-982e-f808b2e68447 None admin] PATH_INFO: `/v3/users/a1072af014f9475fb69143d950c4097f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:17.478583 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b13e49-d7fd-465e-982e-f808b2e68447 None admin] RBAC: Authorizing `identity:get_user(user_id=a1072af014f9475fb69143d950c4097f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:17.480012 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b13e49-d7fd-465e-982e-f808b2e68447 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:17.481617 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 575/1143] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:30:17 2022] GET /identity/v3/users/a1072af014f9475fb69143d950c4097f => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:17.488123 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87711dd9-bc85-4da2-8ba7-6334a82852eb None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:17.589205 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:17.592265 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87711dd9-bc85-4da2-8ba7-6334a82852eb None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:17.625185 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544cf357-ec9e-49dd-ba1b-7402de61a180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:17.625782 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544cf357-ec9e-49dd-ba1b-7402de61a180 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:17.626090 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544cf357-ec9e-49dd-ba1b-7402de61a180 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:17.626361 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544cf357-ec9e-49dd-ba1b-7402de61a180 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:17.693308 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544cf357-ec9e-49dd-ba1b-7402de61a180 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:17.694775 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544cf357-ec9e-49dd-ba1b-7402de61a180 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:17.784211 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:17.787847 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 569/1144] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:30:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 15:30:17.793888 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b13e49-d7fd-465e-982e-f808b2e68447 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:17.864601 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:17.911000 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b13e49-d7fd-465e-982e-f808b2e68447 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:17.945334 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1760b2-bd3a-4d1b-a849-5d9e25999675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:17.946001 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1760b2-bd3a-4d1b-a849-5d9e25999675 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:17.946286 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1760b2-bd3a-4d1b-a849-5d9e25999675 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:17.946559 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1760b2-bd3a-4d1b-a849-5d9e25999675 None admin] PATH_INFO: `/v3/projects/5a598b84c8984260be7f2dc0d1020bf2` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:17.947658 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1760b2-bd3a-4d1b-a849-5d9e25999675 None admin] RBAC: Authorizing `identity:get_project(project_id=5a598b84c8984260be7f2dc0d1020bf2)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:17.949090 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1760b2-bd3a-4d1b-a849-5d9e25999675 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:17.950738 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 576/1145] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:30:17 2022] GET /identity/v3/projects/5a598b84c8984260be7f2dc0d1020bf2 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:17.956872 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544cf357-ec9e-49dd-ba1b-7402de61a180 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:18.055398 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544cf357-ec9e-49dd-ba1b-7402de61a180 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:18.088290 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd256f0f-4589-41e1-ad8c-0fcf9338f9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:18.088878 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd256f0f-4589-41e1-ad8c-0fcf9338f9fc None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:18.089148 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd256f0f-4589-41e1-ad8c-0fcf9338f9fc None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:18.089411 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd256f0f-4589-41e1-ad8c-0fcf9338f9fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:18.090149 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd256f0f-4589-41e1-ad8c-0fcf9338f9fc None admin] RBAC: Adding query filter params (role.id=62d96ac9513c452894fc67b442cfc59f, scope.project.id=5a598b84c8984260be7f2dc0d1020bf2, user.id=a1072af014f9475fb69143d950c4097f) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:18.090457 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd256f0f-4589-41e1-ad8c-0fcf9338f9fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:18.091850 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd256f0f-4589-41e1-ad8c-0fcf9338f9fc None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:18.097558 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 570/1146] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 15:30:17 2022] GET /identity/v3/role_assignments?user.id=a1072af014f9475fb69143d950c4097f&scope.project.id=5a598b84c8984260be7f2dc0d1020bf2&role.id=62d96ac9513c452894fc67b442cfc59f => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:20.203564 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf8d38a-cfd8-4418-a2f7-268c50bf9343 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:20.204081 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf8d38a-cfd8-4418-a2f7-268c50bf9343 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:20.204081 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf8d38a-cfd8-4418-a2f7-268c50bf9343 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:20.205442 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 577/1147] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:30:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:20.210668 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb22a2-1326-4e0d-870e-01fa46c0d2c1 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:20.211064 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb22a2-1326-4e0d-870e-01fa46c0d2c1 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:20.211209 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb22a2-1326-4e0d-870e-01fa46c0d2c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:20.229942 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-9edb22a2-1326-4e0d-870e-01fa46c0d2c1 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:20.231134 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-9edb22a2-1326-4e0d-870e-01fa46c0d2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:20.291069 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:20.300299 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 571/1148] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:30:20.573538 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf8d38a-cfd8-4418-a2f7-268c50bf9343 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:20.677581 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf8d38a-cfd8-4418-a2f7-268c50bf9343 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:20.702446 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:20.712127 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2496c027-9e65-4407-b951-8a04bf9cb463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:20.712752 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2496c027-9e65-4407-b951-8a04bf9cb463 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:20.713036 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2496c027-9e65-4407-b951-8a04bf9cb463 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:20.713309 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2496c027-9e65-4407-b951-8a04bf9cb463 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:20.723060 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2496c027-9e65-4407-b951-8a04bf9cb463 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:30:20.727838 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-2496c027-9e65-4407-b951-8a04bf9cb463 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:30:20.727838 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:20.727838 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:20.727838 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:20.727838 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:20.727838 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:20.727838 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:20.727838 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:20.727838 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:20.727838 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:20.727838 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:20.727838 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:20.727838 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:30:20.727838 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:30:20.727838 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:30:20.727838 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:30:20.728923 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:30:20.728923 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:30:20.728923 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:30:20.728923 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:30:20.728923 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:20.728923 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:20.728923 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:20.728923 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:20.728923 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:20.728923 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:20.728923 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:20.728923 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:20.728923 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:20.728923 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:20.728923 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:20.728923 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:20.730155 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:20.730155 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:20.730155 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:20.730155 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:20.730155 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:30:20.730155 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:30:20.730155 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:30:20.730155 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:30:20.730155 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:30:20.730155 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:30:20.730155 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:30:20.730155 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:20.730155 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 578/1149] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 15:30:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:20.735027 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edb22a2-1326-4e0d-870e-01fa46c0d2c1 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:20.827810 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edb22a2-1326-4e0d-870e-01fa46c0d2c1 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:20.859786 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80e6582-e6b5-4777-b6bf-b5f791de056b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:20.860371 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e6582-e6b5-4777-b6bf-b5f791de056b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:20.860641 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e6582-e6b5-4777-b6bf-b5f791de056b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:20.860903 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e6582-e6b5-4777-b6bf-b5f791de056b None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:20.861541 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80e6582-e6b5-4777-b6bf-b5f791de056b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:20.861880 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80e6582-e6b5-4777-b6bf-b5f791de056b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:20.863222 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80e6582-e6b5-4777-b6bf-b5f791de056b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:20.870812 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 572/1150] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:30:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:20.876593 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2496c027-9e65-4407-b951-8a04bf9cb463 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:20.976346 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2496c027-9e65-4407-b951-8a04bf9cb463 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:21.009628 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe5ba7c-9e67-45e3-9c6f-aef6fd06af45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:21.010273 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5ba7c-9e67-45e3-9c6f-aef6fd06af45 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:21.010553 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5ba7c-9e67-45e3-9c6f-aef6fd06af45 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:21.010824 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe5ba7c-9e67-45e3-9c6f-aef6fd06af45 None admin] PATH_INFO: `/v3/users/a1072af014f9475fb69143d950c4097f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:21.011915 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe5ba7c-9e67-45e3-9c6f-aef6fd06af45 None admin] RBAC: Authorizing `identity:get_user(user_id=a1072af014f9475fb69143d950c4097f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:21.013351 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe5ba7c-9e67-45e3-9c6f-aef6fd06af45 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:21.014999 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 579/1151] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:30:20 2022] GET /identity/v3/users/a1072af014f9475fb69143d950c4097f => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:21.020918 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80e6582-e6b5-4777-b6bf-b5f791de056b None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:21.074908 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:21.118953 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80e6582-e6b5-4777-b6bf-b5f791de056b None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:21.153542 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd696087-5cda-4cb2-90a0-edb84e5fe513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:21.154202 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd696087-5cda-4cb2-90a0-edb84e5fe513 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:21.154478 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd696087-5cda-4cb2-90a0-edb84e5fe513 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:21.154745 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd696087-5cda-4cb2-90a0-edb84e5fe513 None admin] PATH_INFO: `/v3/projects/5a598b84c8984260be7f2dc0d1020bf2` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:21.155836 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd696087-5cda-4cb2-90a0-edb84e5fe513 None admin] RBAC: Authorizing `identity:get_project(project_id=5a598b84c8984260be7f2dc0d1020bf2)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:21.157253 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd696087-5cda-4cb2-90a0-edb84e5fe513 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:21.162011 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 573/1152] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:30:21 2022] GET /identity/v3/projects/5a598b84c8984260be7f2dc0d1020bf2 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:21.164994 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe5ba7c-9e67-45e3-9c6f-aef6fd06af45 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:21.220045 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:21.265738 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe5ba7c-9e67-45e3-9c6f-aef6fd06af45 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:21.300110 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc38d73-3d2c-4398-9964-75b32d5a1fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:21.300845 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc38d73-3d2c-4398-9964-75b32d5a1fb8 None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:21.301134 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc38d73-3d2c-4398-9964-75b32d5a1fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:21.301665 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc38d73-3d2c-4398-9964-75b32d5a1fb8 None admin] PATH_INFO: `/v3/projects/5a598b84c8984260be7f2dc0d1020bf2/users/a1072af014f9475fb69143d950c4097f/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:21.303603 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc38d73-3d2c-4398-9964-75b32d5a1fb8 None admin] RBAC: Authorizing `identity:create_grant(project_id=5a598b84c8984260be7f2dc0d1020bf2, user_id=a1072af014f9475fb69143d950c4097f, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:21.305034 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc38d73-3d2c-4398-9964-75b32d5a1fb8 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:21.319498 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 580/1153] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 15:30:21 2022] PUT /identity/v3/projects/5a598b84c8984260be7f2dc0d1020bf2/users/a1072af014f9475fb69143d950c4097f/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:30:23.445256 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068fb18-2593-49a1-a0b4-05e83b1e19e8 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:23.445822 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068fb18-2593-49a1-a0b4-05e83b1e19e8 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:23.445822 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068fb18-2593-49a1-a0b4-05e83b1e19e8 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:23.447168 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 574/1154] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:30:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:23.452349 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a8065-6361-4365-bfe6-0006c21e740d None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:23.452773 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a8065-6361-4365-bfe6-0006c21e740d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:23.452901 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a8065-6361-4365-bfe6-0006c21e740d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:23.471093 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-504a8065-6361-4365-bfe6-0006c21e740d None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:23.472301 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-504a8065-6361-4365-bfe6-0006c21e740d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:23.536829 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 581/1155] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 15:30:23.818114 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7068fb18-2593-49a1-a0b4-05e83b1e19e8 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:23.917664 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7068fb18-2593-49a1-a0b4-05e83b1e19e8 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:23.940234 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:23.950564 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a4a94a-4a86-41a2-8ee9-e6b7234fe941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:23.951182 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a4a94a-4a86-41a2-8ee9-e6b7234fe941 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:23.951455 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a4a94a-4a86-41a2-8ee9-e6b7234fe941 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:23.951724 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a4a94a-4a86-41a2-8ee9-e6b7234fe941 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:23.961329 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a4a94a-4a86-41a2-8ee9-e6b7234fe941 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:30:23.966061 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-76a4a94a-4a86-41a2-8ee9-e6b7234fe941 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:30:23.966061 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:23.966061 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:23.966061 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:23.966061 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:23.966061 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:23.966061 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:23.966061 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:23.966061 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:23.966061 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:23.966061 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:23.966061 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:23.966061 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:30:23.966061 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:30:23.966061 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:30:23.966061 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:30:23.967211 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:30:23.967211 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:30:23.967211 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:30:23.967211 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:30:23.967211 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:23.967211 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:23.967211 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:23.967211 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:23.967211 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:23.967211 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:23.967211 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:23.967211 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:23.967211 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:23.967211 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:23.967211 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:23.967211 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:23.968282 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:23.968282 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:23.968282 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:23.968282 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:23.968282 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:30:23.968282 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:30:23.968282 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:30:23.968282 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:30:23.968282 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:30:23.968282 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:30:23.968282 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:30:23.968282 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:23.968282 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 575/1156] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 15:30:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:23.973971 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504a8065-6361-4365-bfe6-0006c21e740d None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:24.022206 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:24.074486 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504a8065-6361-4365-bfe6-0006c21e740d None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:24.107213 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244f633c-2d52-452d-a815-aa59913d442a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:24.107816 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f633c-2d52-452d-a815-aa59913d442a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:24.108095 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f633c-2d52-452d-a815-aa59913d442a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:24.108367 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f633c-2d52-452d-a815-aa59913d442a None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:24.109020 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244f633c-2d52-452d-a815-aa59913d442a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:24.109319 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244f633c-2d52-452d-a815-aa59913d442a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:24.110717 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244f633c-2d52-452d-a815-aa59913d442a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:24.118433 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 582/1157] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:30:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:24.124389 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a4a94a-4a86-41a2-8ee9-e6b7234fe941 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:24.219793 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a4a94a-4a86-41a2-8ee9-e6b7234fe941 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:24.252176 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce4f84f-f8cf-4e6b-8a1a-6e8345fdf70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:24.252777 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce4f84f-f8cf-4e6b-8a1a-6e8345fdf70e None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:24.253050 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce4f84f-f8cf-4e6b-8a1a-6e8345fdf70e None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:24.253570 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce4f84f-f8cf-4e6b-8a1a-6e8345fdf70e None admin] PATH_INFO: `/v3/users/a1072af014f9475fb69143d950c4097f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:24.254683 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce4f84f-f8cf-4e6b-8a1a-6e8345fdf70e None admin] RBAC: Authorizing `identity:get_user(user_id=a1072af014f9475fb69143d950c4097f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:24.256082 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce4f84f-f8cf-4e6b-8a1a-6e8345fdf70e None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:24.257619 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 576/1158] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:30:24 2022] GET /identity/v3/users/a1072af014f9475fb69143d950c4097f => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:24.263432 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244f633c-2d52-452d-a815-aa59913d442a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:24.371546 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244f633c-2d52-452d-a815-aa59913d442a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:24.379006 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:24.405524 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6a639e-fed8-4aa7-a1fc-4fb528477054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:24.406144 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a639e-fed8-4aa7-a1fc-4fb528477054 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:24.406419 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a639e-fed8-4aa7-a1fc-4fb528477054 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:24.406683 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a639e-fed8-4aa7-a1fc-4fb528477054 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:24.471217 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6a639e-fed8-4aa7-a1fc-4fb528477054 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:24.472648 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6a639e-fed8-4aa7-a1fc-4fb528477054 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:24.568069 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 583/1159] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:30:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 15:30:24.574192 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce4f84f-f8cf-4e6b-8a1a-6e8345fdf70e None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:24.625967 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:24.674566 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce4f84f-f8cf-4e6b-8a1a-6e8345fdf70e None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:24.708209 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41759d56-f3a9-45b1-8613-a710bdae20e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:24.708826 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41759d56-f3a9-45b1-8613-a710bdae20e0 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:24.709101 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41759d56-f3a9-45b1-8613-a710bdae20e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:24.709366 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41759d56-f3a9-45b1-8613-a710bdae20e0 None admin] PATH_INFO: `/v3/projects/5a598b84c8984260be7f2dc0d1020bf2` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:24.710467 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41759d56-f3a9-45b1-8613-a710bdae20e0 None admin] RBAC: Authorizing `identity:get_project(project_id=5a598b84c8984260be7f2dc0d1020bf2)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:24.711876 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41759d56-f3a9-45b1-8613-a710bdae20e0 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:24.713479 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 577/1160] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:30:24 2022] GET /identity/v3/projects/5a598b84c8984260be7f2dc0d1020bf2 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:24.719666 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6a639e-fed8-4aa7-a1fc-4fb528477054 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:24.727542 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:24.813628 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6a639e-fed8-4aa7-a1fc-4fb528477054 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:24.845730 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97efadec-59bf-499f-9a85-5501bef9fcbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:24.846390 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97efadec-59bf-499f-9a85-5501bef9fcbe None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:24.846665 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97efadec-59bf-499f-9a85-5501bef9fcbe None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:24.846932 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97efadec-59bf-499f-9a85-5501bef9fcbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:24.847631 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97efadec-59bf-499f-9a85-5501bef9fcbe None admin] RBAC: Adding query filter params (role.id=62d96ac9513c452894fc67b442cfc59f, scope.project.id=5a598b84c8984260be7f2dc0d1020bf2, user.id=a1072af014f9475fb69143d950c4097f) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:24.847931 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97efadec-59bf-499f-9a85-5501bef9fcbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:24.849336 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97efadec-59bf-499f-9a85-5501bef9fcbe None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:24.854833 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 584/1161] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 15:30:24 2022] GET /identity/v3/role_assignments?user.id=a1072af014f9475fb69143d950c4097f&scope.project.id=5a598b84c8984260be7f2dc0d1020bf2&role.id=62d96ac9513c452894fc67b442cfc59f => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:27.007761 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e83fdf-4fda-4a46-bd61-3d55581f24ee None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:27.008308 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e83fdf-4fda-4a46-bd61-3d55581f24ee None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:27.008308 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e83fdf-4fda-4a46-bd61-3d55581f24ee None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:27.009617 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 578/1162] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:30:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:27.014712 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e7061d-0b8f-496b-949b-64bf01aa8845 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:27.015151 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e7061d-0b8f-496b-949b-64bf01aa8845 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:27.015271 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e7061d-0b8f-496b-949b-64bf01aa8845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:27.034115 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-c8e7061d-0b8f-496b-949b-64bf01aa8845 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:27.035286 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c8e7061d-0b8f-496b-949b-64bf01aa8845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:27.094949 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:27.100604 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 585/1163] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:30:27.365414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e83fdf-4fda-4a46-bd61-3d55581f24ee None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:27.466548 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e83fdf-4fda-4a46-bd61-3d55581f24ee None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:27.486113 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:27.500717 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f707383a-a291-44a9-8483-31d7e7e186c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:27.501309 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707383a-a291-44a9-8483-31d7e7e186c8 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:27.501581 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707383a-a291-44a9-8483-31d7e7e186c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:27.501870 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707383a-a291-44a9-8483-31d7e7e186c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:27.565340 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f707383a-a291-44a9-8483-31d7e7e186c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:27.566784 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f707383a-a291-44a9-8483-31d7e7e186c8 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:27.660255 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 579/1164] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:30:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 28 15:30:27.666164 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e7061d-0b8f-496b-949b-64bf01aa8845 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:27.762567 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e7061d-0b8f-496b-949b-64bf01aa8845 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:27.794350 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be2877a-0436-4497-b6a1-0c563cc7c1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:27.794975 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be2877a-0436-4497-b6a1-0c563cc7c1af None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:27.795259 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be2877a-0436-4497-b6a1-0c563cc7c1af None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:27.795534 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be2877a-0436-4497-b6a1-0c563cc7c1af None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:27.802502 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be2877a-0436-4497-b6a1-0c563cc7c1af None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:27.804011 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be2877a-0436-4497-b6a1-0c563cc7c1af None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:27.816005 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-6be2877a-0436-4497-b6a1-0c563cc7c1af None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 28 15:30:27.816005 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:27.816005 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 28 15:30:27.816005 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 28 15:30:27.816005 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:30:27.816005 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:30:27.816005 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:30:27.816005 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:30:27.816005 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 28 15:30:27.816005 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:27.816005 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:30:27.816005 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:27.816005 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:27.816005 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:27.816005 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:27.816005 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:27.816005 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:27.817218 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:27.817218 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:27.817218 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:27.817218 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:27.817218 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:27.817218 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:27.817218 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:30:27.817218 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:30:27.817218 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:30:27.817218 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:30:27.817218 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:27.817218 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:27.817218 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:27.817218 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:27.817218 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:27.817218 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:27.818125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:27.818125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:27.818125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:27.818125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:27.818125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:27.818125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:27.818125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:27.818125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:27.818125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:27.818125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:27.818125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 28 15:30:27.818125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:30:27.818125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 28 15:30:27.818125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:27.818125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 586/1165] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Sep 28 15:30:27 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:27.823249 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f707383a-a291-44a9-8483-31d7e7e186c8 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:27.843647 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:27.919068 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f707383a-a291-44a9-8483-31d7e7e186c8 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:27.952018 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0d81ad-cd6a-4e57-bd50-5b369fb0605d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:27.952610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d81ad-cd6a-4e57-bd50-5b369fb0605d None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:27.952886 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d81ad-cd6a-4e57-bd50-5b369fb0605d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:27.953153 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d81ad-cd6a-4e57-bd50-5b369fb0605d None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:27.953778 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0d81ad-cd6a-4e57-bd50-5b369fb0605d None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:27.954105 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0d81ad-cd6a-4e57-bd50-5b369fb0605d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:27.955478 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0d81ad-cd6a-4e57-bd50-5b369fb0605d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:27.962283 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 580/1166] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Sep 28 15:30:27 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:30.060380 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43204a5-499e-48c2-849d-9398b2281bf3 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:30.060926 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43204a5-499e-48c2-849d-9398b2281bf3 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:30.060926 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43204a5-499e-48c2-849d-9398b2281bf3 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:30.062305 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 587/1167] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:30:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:30.067603 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809c3a7-2776-4cd4-bff9-bf7b273920c6 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:30.068027 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809c3a7-2776-4cd4-bff9-bf7b273920c6 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:30.068146 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809c3a7-2776-4cd4-bff9-bf7b273920c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:30.086164 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-f809c3a7-2776-4cd4-bff9-bf7b273920c6 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:30.087310 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-f809c3a7-2776-4cd4-bff9-bf7b273920c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:30.150125 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 581/1168] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:30:30.415577 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43204a5-499e-48c2-849d-9398b2281bf3 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:30.474190 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:30.514378 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43204a5-499e-48c2-849d-9398b2281bf3 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:30.547304 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01409f75-a096-498a-8db5-47b9e9360310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:30.547888 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01409f75-a096-498a-8db5-47b9e9360310 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:30.548164 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01409f75-a096-498a-8db5-47b9e9360310 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:30.548434 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01409f75-a096-498a-8db5-47b9e9360310 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:30.549105 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01409f75-a096-498a-8db5-47b9e9360310 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:30.550492 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01409f75-a096-498a-8db5-47b9e9360310 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:30.566891 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 588/1169] 10.222.0.52 () {60 vars in 1138 bytes} [Wed Sep 28 15:30:30 2022] POST /identity/v3/domains => generated 267 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:30:32.713715 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747f782-b0d0-4a6f-9603-5272230cc634 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:32.714278 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747f782-b0d0-4a6f-9603-5272230cc634 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:32.714278 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747f782-b0d0-4a6f-9603-5272230cc634 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:32.715630 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 582/1170] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:30:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:32.721351 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbdc234-8442-4754-8510-beb7418ab280 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:32.721769 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbdc234-8442-4754-8510-beb7418ab280 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:32.721923 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbdc234-8442-4754-8510-beb7418ab280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:32.740055 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-dcbdc234-8442-4754-8510-beb7418ab280 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:32.741219 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-dcbdc234-8442-4754-8510-beb7418ab280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:32.805280 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 589/1171] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:30:33.070186 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f747f782-b0d0-4a6f-9603-5272230cc634 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:33.086089 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:33.170192 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f747f782-b0d0-4a6f-9603-5272230cc634 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:33.202314 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9936dfc5-a526-496d-be12-090a6c584406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:33.202930 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936dfc5-a526-496d-be12-090a6c584406 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:33.203206 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936dfc5-a526-496d-be12-090a6c584406 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:33.203470 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936dfc5-a526-496d-be12-090a6c584406 None admin] PATH_INFO: `/v3/domains/2b43f20ba4244a849a157b727fed0c8a` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:33.214536 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9936dfc5-a526-496d-be12-090a6c584406 None admin] RBAC: Authorizing `identity:get_domain(domain_id=2b43f20ba4244a849a157b727fed0c8a)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:33.215957 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9936dfc5-a526-496d-be12-090a6c584406 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:33.217558 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 583/1172] 10.222.0.52 () {56 vars in 1183 bytes} [Wed Sep 28 15:30:33 2022] GET /identity/v3/domains/2b43f20ba4244a849a157b727fed0c8a => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:33.224257 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbdc234-8442-4754-8510-beb7418ab280 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:33.271462 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:33.320435 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbdc234-8442-4754-8510-beb7418ab280 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:33.353331 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc019f4c-fa88-4ce9-bc05-1d9420d24464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:33.353963 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc019f4c-fa88-4ce9-bc05-1d9420d24464 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:33.354242 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc019f4c-fa88-4ce9-bc05-1d9420d24464 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:33.354515 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc019f4c-fa88-4ce9-bc05-1d9420d24464 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:33.355159 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc019f4c-fa88-4ce9-bc05-1d9420d24464 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:33.356572 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc019f4c-fa88-4ce9-bc05-1d9420d24464 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:33.382343 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 590/1173] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Sep 28 15:30:33 2022] POST /identity/v3/projects => generated 361 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:30:35.529419 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc4ceb-c8c7-45f7-8d3d-766f3bf83075 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:35.529973 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc4ceb-c8c7-45f7-8d3d-766f3bf83075 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:35.529973 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc4ceb-c8c7-45f7-8d3d-766f3bf83075 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:35.531297 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 584/1174] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:30:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:35.536633 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ccb63-8cd5-4001-8724-7bffefc28a74 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:35.537053 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ccb63-8cd5-4001-8724-7bffefc28a74 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:35.537183 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ccb63-8cd5-4001-8724-7bffefc28a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:35.555348 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-0b7ccb63-8cd5-4001-8724-7bffefc28a74 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:35.556524 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0b7ccb63-8cd5-4001-8724-7bffefc28a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:35.619826 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 591/1175] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:30:35.891797 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfc4ceb-c8c7-45f7-8d3d-766f3bf83075 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:35.964948 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:35.992559 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfc4ceb-c8c7-45f7-8d3d-766f3bf83075 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:36.029527 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eccbd8-3906-4eb9-aabc-b5737a69b665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:36.030665 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eccbd8-3906-4eb9-aabc-b5737a69b665 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:36.031147 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eccbd8-3906-4eb9-aabc-b5737a69b665 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:36.031624 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eccbd8-3906-4eb9-aabc-b5737a69b665 None admin] PATH_INFO: `/v3/domains/2b43f20ba4244a849a157b727fed0c8a` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:36.033639 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eccbd8-3906-4eb9-aabc-b5737a69b665 None admin] RBAC: Authorizing `identity:get_domain(domain_id=2b43f20ba4244a849a157b727fed0c8a)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:36.035064 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eccbd8-3906-4eb9-aabc-b5737a69b665 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:36.036605 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 585/1176] 10.222.0.52 () {56 vars in 1183 bytes} [Wed Sep 28 15:30:35 2022] GET /identity/v3/domains/2b43f20ba4244a849a157b727fed0c8a => generated 267 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:36.042785 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7ccb63-8cd5-4001-8724-7bffefc28a74 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:36.073353 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:36.138994 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7ccb63-8cd5-4001-8724-7bffefc28a74 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:36.171685 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a1d249-a824-4042-8abe-5248886098f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:36.172226 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a1d249-a824-4042-8abe-5248886098f9 None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:36.172500 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a1d249-a824-4042-8abe-5248886098f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:36.172767 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a1d249-a824-4042-8abe-5248886098f9 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:36.173424 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a1d249-a824-4042-8abe-5248886098f9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:36.174870 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a1d249-a824-4042-8abe-5248886098f9 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:36.195669 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 592/1177] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 15:30:36 2022] POST /identity/v3/users => generated 311 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:30:38.361105 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ceab4d-aed9-4799-8711-3272efc3aa10 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:38.361598 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ceab4d-aed9-4799-8711-3272efc3aa10 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:38.361719 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ceab4d-aed9-4799-8711-3272efc3aa10 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:38.362970 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 586/1178] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:30:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:38.368102 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ccf56-7231-4cef-b441-8b8eaba40e7b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:38.368534 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ccf56-7231-4cef-b441-8b8eaba40e7b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:38.368712 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ccf56-7231-4cef-b441-8b8eaba40e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:38.387297 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-951ccf56-7231-4cef-b441-8b8eaba40e7b None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:38.388476 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-951ccf56-7231-4cef-b441-8b8eaba40e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:38.455834 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 593/1179] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:30:38.725123 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ceab4d-aed9-4799-8711-3272efc3aa10 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:38.826475 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ceab4d-aed9-4799-8711-3272efc3aa10 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:38.859939 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84734867-31d3-49e4-9c23-e5b6c51b7dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:38.861108 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84734867-31d3-49e4-9c23-e5b6c51b7dc6 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:38.861383 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84734867-31d3-49e4-9c23-e5b6c51b7dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:38.861652 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84734867-31d3-49e4-9c23-e5b6c51b7dc6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:38.871741 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84734867-31d3-49e4-9c23-e5b6c51b7dc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:30:38.876579 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-84734867-31d3-49e4-9c23-e5b6c51b7dc6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:30:38.876579 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:38.876579 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:38.876579 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:38.876579 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:38.876579 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:38.876579 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:38.876579 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:38.876579 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:38.876579 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:38.876579 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:38.876579 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:38.876579 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:30:38.876579 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:30:38.876579 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:30:38.876579 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:30:38.877536 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:30:38.877536 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:30:38.877536 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:30:38.877536 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:30:38.877536 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:38.877536 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:38.877536 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:38.877536 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:38.877536 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:38.877536 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:38.877536 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:38.877536 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:38.877536 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:38.877536 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:38.877536 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:38.877536 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:38.878728 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:38.878728 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:38.878728 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:38.878728 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:38.878728 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:30:38.878728 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:30:38.878728 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:30:38.878728 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:30:38.878728 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:30:38.878728 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:30:38.878728 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:30:38.878728 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:38.882916 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 587/1180] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 15:30:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:38.884071 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951ccf56-7231-4cef-b441-8b8eaba40e7b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:38.903895 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:38.989644 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951ccf56-7231-4cef-b441-8b8eaba40e7b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:39.025484 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72317b9-8a7a-47e5-b5af-9da1ff182847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:39.026125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72317b9-8a7a-47e5-b5af-9da1ff182847 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:39.026404 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72317b9-8a7a-47e5-b5af-9da1ff182847 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:39.026675 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72317b9-8a7a-47e5-b5af-9da1ff182847 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:39.027319 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72317b9-8a7a-47e5-b5af-9da1ff182847 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:39.027618 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72317b9-8a7a-47e5-b5af-9da1ff182847 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:39.028979 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72317b9-8a7a-47e5-b5af-9da1ff182847 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:39.037439 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 594/1181] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:30:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:39.043488 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84734867-31d3-49e4-9c23-e5b6c51b7dc6 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:39.046896 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:39.143399 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84734867-31d3-49e4-9c23-e5b6c51b7dc6 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:39.177929 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830df46e-35ee-4c39-947d-e1d6bd9b6ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:39.178573 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830df46e-35ee-4c39-947d-e1d6bd9b6ced None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:39.178850 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830df46e-35ee-4c39-947d-e1d6bd9b6ced None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:39.179122 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830df46e-35ee-4c39-947d-e1d6bd9b6ced None admin] PATH_INFO: `/v3/users/f093c8a08cd44964b10a5c083fda2d79` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:39.192444 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830df46e-35ee-4c39-947d-e1d6bd9b6ced None admin] RBAC: Authorizing `identity:get_user(user_id=f093c8a08cd44964b10a5c083fda2d79)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:39.193918 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830df46e-35ee-4c39-947d-e1d6bd9b6ced None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:39.195536 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 588/1182] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:30:39 2022] GET /identity/v3/users/f093c8a08cd44964b10a5c083fda2d79 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:39.201289 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72317b9-8a7a-47e5-b5af-9da1ff182847 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:39.285072 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:39.304690 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72317b9-8a7a-47e5-b5af-9da1ff182847 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:39.340690 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa501153-4cc1-4969-a1f4-1285311ab877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:39.341295 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa501153-4cc1-4969-a1f4-1285311ab877 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:39.341572 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa501153-4cc1-4969-a1f4-1285311ab877 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:39.341839 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa501153-4cc1-4969-a1f4-1285311ab877 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:39.415435 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa501153-4cc1-4969-a1f4-1285311ab877 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:39.416879 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa501153-4cc1-4969-a1f4-1285311ab877 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:39.497499 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:39.517495 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 595/1183] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:30:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 15:30:39.523541 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830df46e-35ee-4c39-947d-e1d6bd9b6ced None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:39.583222 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:39.624302 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830df46e-35ee-4c39-947d-e1d6bd9b6ced None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:39.659310 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dfed72-2ffd-4642-aeae-4ae79fc58b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:39.659928 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dfed72-2ffd-4642-aeae-4ae79fc58b4e None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:39.660199 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dfed72-2ffd-4642-aeae-4ae79fc58b4e None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:39.660464 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dfed72-2ffd-4642-aeae-4ae79fc58b4e None admin] PATH_INFO: `/v3/projects/6ba27e6e611a49cea4a0ba6b0fe666a0` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:39.661553 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dfed72-2ffd-4642-aeae-4ae79fc58b4e None admin] RBAC: Authorizing `identity:get_project(project_id=6ba27e6e611a49cea4a0ba6b0fe666a0)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:39.662989 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dfed72-2ffd-4642-aeae-4ae79fc58b4e None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:39.664587 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 589/1184] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:30:39 2022] GET /identity/v3/projects/6ba27e6e611a49cea4a0ba6b0fe666a0 => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:39.670777 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa501153-4cc1-4969-a1f4-1285311ab877 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:39.772909 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa501153-4cc1-4969-a1f4-1285311ab877 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:39.807259 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ae8fc6-086f-4c5a-a4e2-2276ecb28a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:39.807858 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae8fc6-086f-4c5a-a4e2-2276ecb28a12 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:39.808146 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae8fc6-086f-4c5a-a4e2-2276ecb28a12 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:39.808418 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae8fc6-086f-4c5a-a4e2-2276ecb28a12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:39.809128 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ae8fc6-086f-4c5a-a4e2-2276ecb28a12 None admin] RBAC: Adding query filter params (role.id=62d96ac9513c452894fc67b442cfc59f, scope.project.id=6ba27e6e611a49cea4a0ba6b0fe666a0, user.id=f093c8a08cd44964b10a5c083fda2d79) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:39.809431 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ae8fc6-086f-4c5a-a4e2-2276ecb28a12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:39.810881 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ae8fc6-086f-4c5a-a4e2-2276ecb28a12 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:39.817890 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 596/1185] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 15:30:39 2022] GET /identity/v3/role_assignments?user.id=f093c8a08cd44964b10a5c083fda2d79&scope.project.id=6ba27e6e611a49cea4a0ba6b0fe666a0&role.id=62d96ac9513c452894fc67b442cfc59f => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:41.953451 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f87ee8-08c3-4556-8658-c89f1f29c2dd None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:41.953451 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f87ee8-08c3-4556-8658-c89f1f29c2dd None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:41.953451 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f87ee8-08c3-4556-8658-c89f1f29c2dd None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:41.953451 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 590/1186] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:30:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:41.958791 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a2c537-4756-48b8-9ec3-5319a6ee0b38 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:41.959225 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a2c537-4756-48b8-9ec3-5319a6ee0b38 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:41.961893 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a2c537-4756-48b8-9ec3-5319a6ee0b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:41.978553 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-b0a2c537-4756-48b8-9ec3-5319a6ee0b38 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:41.979715 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b0a2c537-4756-48b8-9ec3-5319a6ee0b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:42.020743 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:42.047966 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 597/1187] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:30:42.315720 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f87ee8-08c3-4556-8658-c89f1f29c2dd None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:42.419281 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f87ee8-08c3-4556-8658-c89f1f29c2dd None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:42.446749 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:42.454684 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e2e668-8ae1-48de-8401-1ab43e1cfc32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:42.455301 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e2e668-8ae1-48de-8401-1ab43e1cfc32 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:42.455572 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e2e668-8ae1-48de-8401-1ab43e1cfc32 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:42.455835 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e2e668-8ae1-48de-8401-1ab43e1cfc32 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:42.465966 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e2e668-8ae1-48de-8401-1ab43e1cfc32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:30:42.471258 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-19e2e668-8ae1-48de-8401-1ab43e1cfc32 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:30:42.471258 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:42.471258 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:42.471258 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:42.471258 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:42.471258 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:42.471258 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:42.471258 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:42.471258 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:42.471258 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:42.471258 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:42.471258 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:42.471258 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:30:42.471258 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:30:42.471258 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:30:42.471258 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:30:42.472287 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:30:42.472287 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:30:42.472287 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:30:42.472287 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:30:42.472287 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:42.472287 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:42.472287 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:42.472287 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:42.472287 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:42.472287 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:42.472287 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:42.472287 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:42.472287 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:42.472287 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:42.472287 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:42.472287 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:42.473153 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:42.473153 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:42.473153 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:42.473153 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:42.473153 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:30:42.473153 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:30:42.473153 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:30:42.473153 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:30:42.473153 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:30:42.473153 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:30:42.473153 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:30:42.473153 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:30:42.473153 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 591/1188] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 15:30:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:42.478429 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a2c537-4756-48b8-9ec3-5319a6ee0b38 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:42.577127 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a2c537-4756-48b8-9ec3-5319a6ee0b38 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:42.611658 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea44023-a355-4743-a961-ae8eef9e764d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:42.612265 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea44023-a355-4743-a961-ae8eef9e764d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:42.612546 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea44023-a355-4743-a961-ae8eef9e764d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:42.612814 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea44023-a355-4743-a961-ae8eef9e764d None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:42.613467 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea44023-a355-4743-a961-ae8eef9e764d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:42.613773 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea44023-a355-4743-a961-ae8eef9e764d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:42.615173 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea44023-a355-4743-a961-ae8eef9e764d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:42.623459 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 598/1189] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:30:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:42.630154 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e2e668-8ae1-48de-8401-1ab43e1cfc32 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:42.733057 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e2e668-8ae1-48de-8401-1ab43e1cfc32 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:42.770466 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9c519b-68ef-41ee-969c-f24df825bd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:42.771086 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c519b-68ef-41ee-969c-f24df825bd03 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:42.771360 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c519b-68ef-41ee-969c-f24df825bd03 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:42.771628 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c519b-68ef-41ee-969c-f24df825bd03 None admin] PATH_INFO: `/v3/users/f093c8a08cd44964b10a5c083fda2d79` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:42.772744 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9c519b-68ef-41ee-969c-f24df825bd03 None admin] RBAC: Authorizing `identity:get_user(user_id=f093c8a08cd44964b10a5c083fda2d79)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:42.774384 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9c519b-68ef-41ee-969c-f24df825bd03 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:42.775938 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 592/1190] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:30:42 2022] GET /identity/v3/users/f093c8a08cd44964b10a5c083fda2d79 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:42.781647 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea44023-a355-4743-a961-ae8eef9e764d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:42.818744 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:42.886428 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea44023-a355-4743-a961-ae8eef9e764d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:42.922185 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5da24d-153c-487d-9b46-6d8e2e547ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:42.922844 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5da24d-153c-487d-9b46-6d8e2e547ab2 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:42.923119 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5da24d-153c-487d-9b46-6d8e2e547ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:42.923384 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5da24d-153c-487d-9b46-6d8e2e547ab2 None admin] PATH_INFO: `/v3/projects/6ba27e6e611a49cea4a0ba6b0fe666a0` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:42.924593 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5da24d-153c-487d-9b46-6d8e2e547ab2 None admin] RBAC: Authorizing `identity:get_project(project_id=6ba27e6e611a49cea4a0ba6b0fe666a0)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:42.926131 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5da24d-153c-487d-9b46-6d8e2e547ab2 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:42.927917 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 599/1191] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:30:42 2022] GET /identity/v3/projects/6ba27e6e611a49cea4a0ba6b0fe666a0 => generated 361 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:42.934298 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9c519b-68ef-41ee-969c-f24df825bd03 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:42.994215 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:43.041237 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9c519b-68ef-41ee-969c-f24df825bd03 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:43.078654 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77711b1-1382-4370-b231-957779ae9cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:43.079795 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77711b1-1382-4370-b231-957779ae9cc2 None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:43.080072 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77711b1-1382-4370-b231-957779ae9cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:43.080338 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77711b1-1382-4370-b231-957779ae9cc2 None admin] PATH_INFO: `/v3/projects/6ba27e6e611a49cea4a0ba6b0fe666a0/users/f093c8a08cd44964b10a5c083fda2d79/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:43.082296 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77711b1-1382-4370-b231-957779ae9cc2 None admin] RBAC: Authorizing `identity:create_grant(project_id=6ba27e6e611a49cea4a0ba6b0fe666a0, user_id=f093c8a08cd44964b10a5c083fda2d79, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:43.083716 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77711b1-1382-4370-b231-957779ae9cc2 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:43.097455 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 593/1192] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 15:30:42 2022] PUT /identity/v3/projects/6ba27e6e611a49cea4a0ba6b0fe666a0/users/f093c8a08cd44964b10a5c083fda2d79/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:30:45.227040 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a1001-9a8c-400c-a8e2-d04b198438be None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:45.227541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a1001-9a8c-400c-a8e2-d04b198438be None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:45.227630 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a1001-9a8c-400c-a8e2-d04b198438be None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:45.228945 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 600/1193] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:30:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:30:45.234150 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b81ba1-d238-4859-8091-31dba57f45a0 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:45.234524 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b81ba1-d238-4859-8091-31dba57f45a0 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:45.234692 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b81ba1-d238-4859-8091-31dba57f45a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:45.255756 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-71b81ba1-d238-4859-8091-31dba57f45a0 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:30:45.256969 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-71b81ba1-d238-4859-8091-31dba57f45a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:30:45.323622 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 594/1194] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:30:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:30:45.594049 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a1001-9a8c-400c-a8e2-d04b198438be None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:45.692835 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a1001-9a8c-400c-a8e2-d04b198438be None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:45.695117 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:45.726658 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94894fd7-58ba-4c33-a474-9750a4ac8fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:45.727278 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94894fd7-58ba-4c33-a474-9750a4ac8fde None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:45.727553 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94894fd7-58ba-4c33-a474-9750a4ac8fde None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:45.727822 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94894fd7-58ba-4c33-a474-9750a4ac8fde None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:45.738128 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94894fd7-58ba-4c33-a474-9750a4ac8fde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:30:45.742846 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-94894fd7-58ba-4c33-a474-9750a4ac8fde None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:30:45.742846 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:30:45.742846 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:30:45.742846 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:30:45.742846 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:30:45.742846 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:30:45.742846 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:30:45.742846 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:30:45.742846 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:30:45.742846 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:30:45.742846 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:30:45.742846 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:30:45.742846 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:30:45.742846 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:30:45.742846 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:30:45.742846 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:30:45.743949 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:30:45.743949 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:30:45.743949 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:30:45.743949 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:30:45.743949 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:30:45.743949 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:30:45.743949 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:30:45.743949 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:30:45.743949 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:30:45.743949 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:30:45.743949 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:30:45.743949 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:30:45.743949 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:30:45.743949 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:30:45.743949 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:30:45.743949 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:30:45.745133 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:30:45.745133 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:30:45.745133 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:30:45.745133 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:30:45.745133 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:30:45.745133 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:30:45.745133 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:30:45.745133 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:30:45.745133 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:30:45.745133 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:30:45.745133 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:30:45.745133 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:30:45.745133 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 601/1195] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 15:30:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:30:45.749957 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b81ba1-d238-4859-8091-31dba57f45a0 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:45.794679 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:45.845175 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b81ba1-d238-4859-8091-31dba57f45a0 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:45.876661 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306b151a-126f-4ecc-94fe-9e00a52a7f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:45.877244 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b151a-126f-4ecc-94fe-9e00a52a7f33 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:45.877783 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b151a-126f-4ecc-94fe-9e00a52a7f33 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:45.878072 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b151a-126f-4ecc-94fe-9e00a52a7f33 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:45.878714 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306b151a-126f-4ecc-94fe-9e00a52a7f33 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:45.879010 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306b151a-126f-4ecc-94fe-9e00a52a7f33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:45.880344 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306b151a-126f-4ecc-94fe-9e00a52a7f33 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:45.887989 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 595/1196] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:30:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:45.894466 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94894fd7-58ba-4c33-a474-9750a4ac8fde None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:45.990643 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94894fd7-58ba-4c33-a474-9750a4ac8fde None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:46.023499 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1bc102-ab71-4243-933c-aefd089372ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:46.024113 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1bc102-ab71-4243-933c-aefd089372ea None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:46.024389 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1bc102-ab71-4243-933c-aefd089372ea None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:46.024659 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1bc102-ab71-4243-933c-aefd089372ea None admin] PATH_INFO: `/v3/users/f093c8a08cd44964b10a5c083fda2d79` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:46.025752 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1bc102-ab71-4243-933c-aefd089372ea None admin] RBAC: Authorizing `identity:get_user(user_id=f093c8a08cd44964b10a5c083fda2d79)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:46.027200 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1bc102-ab71-4243-933c-aefd089372ea None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:46.028739 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 602/1197] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Sep 28 15:30:45 2022] GET /identity/v3/users/f093c8a08cd44964b10a5c083fda2d79 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:46.035120 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306b151a-126f-4ecc-94fe-9e00a52a7f33 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:46.130589 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306b151a-126f-4ecc-94fe-9e00a52a7f33 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:46.138782 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:46.162884 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942e09b4-09a6-4bd9-bd19-ca263393c3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:46.163468 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942e09b4-09a6-4bd9-bd19-ca263393c3a5 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:46.163738 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942e09b4-09a6-4bd9-bd19-ca263393c3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:46.163998 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942e09b4-09a6-4bd9-bd19-ca263393c3a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:46.226835 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942e09b4-09a6-4bd9-bd19-ca263393c3a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:46.228246 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942e09b4-09a6-4bd9-bd19-ca263393c3a5 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:46.321457 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 596/1198] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:30:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 15:30:46.327275 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1bc102-ab71-4243-933c-aefd089372ea None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:46.357091 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:46.423039 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1bc102-ab71-4243-933c-aefd089372ea None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:46.454594 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51116266-ba50-4756-91fa-ca5e2facbd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:46.455196 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51116266-ba50-4756-91fa-ca5e2facbd68 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:46.455470 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51116266-ba50-4756-91fa-ca5e2facbd68 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:46.455738 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51116266-ba50-4756-91fa-ca5e2facbd68 None admin] PATH_INFO: `/v3/projects/6ba27e6e611a49cea4a0ba6b0fe666a0` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:46.456818 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51116266-ba50-4756-91fa-ca5e2facbd68 None admin] RBAC: Authorizing `identity:get_project(project_id=6ba27e6e611a49cea4a0ba6b0fe666a0)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:46.458269 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51116266-ba50-4756-91fa-ca5e2facbd68 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:46.459853 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 603/1199] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Sep 28 15:30:46 2022] GET /identity/v3/projects/6ba27e6e611a49cea4a0ba6b0fe666a0 => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:46.465910 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942e09b4-09a6-4bd9-bd19-ca263393c3a5 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:30:46.476049 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:46.560675 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942e09b4-09a6-4bd9-bd19-ca263393c3a5 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:30:46.592518 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d56c04-04a2-4d93-8259-b4378aba4f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:30:46.593100 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d56c04-04a2-4d93-8259-b4378aba4f07 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:30:46.593371 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d56c04-04a2-4d93-8259-b4378aba4f07 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:30:46.593633 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d56c04-04a2-4d93-8259-b4378aba4f07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:30:46.594953 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d56c04-04a2-4d93-8259-b4378aba4f07 None admin] RBAC: Adding query filter params (role.id=62d96ac9513c452894fc67b442cfc59f, scope.project.id=6ba27e6e611a49cea4a0ba6b0fe666a0, user.id=f093c8a08cd44964b10a5c083fda2d79) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:30:46.595249 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d56c04-04a2-4d93-8259-b4378aba4f07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:30:46.596644 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d56c04-04a2-4d93-8259-b4378aba4f07 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:30:46.602113 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 597/1200] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 15:30:46 2022] GET /identity/v3/role_assignments?user.id=f093c8a08cd44964b10a5c083fda2d79&scope.project.id=6ba27e6e611a49cea4a0ba6b0fe666a0&role.id=62d96ac9513c452894fc67b442cfc59f => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:30:56.450303 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=50 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:30:56.599614 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=64 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:42.309101 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a09fda-fb24-462f-8a5d-8fe6cff6bb93 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:42.309694 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a09fda-fb24-462f-8a5d-8fe6cff6bb93 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:42.309694 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a09fda-fb24-462f-8a5d-8fe6cff6bb93 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:42.311232 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 604/1201] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:31:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:31:42.320780 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd7cd4a-9409-4839-a7b9-979ccddee145 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:42.320780 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd7cd4a-9409-4839-a7b9-979ccddee145 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:42.320780 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd7cd4a-9409-4839-a7b9-979ccddee145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:42.337568 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-8bd7cd4a-9409-4839-a7b9-979ccddee145 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:31:42.339187 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-8bd7cd4a-9409-4839-a7b9-979ccddee145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:31:42.438015 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 598/1202] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:31:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:31:42.724311 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a09fda-fb24-462f-8a5d-8fe6cff6bb93 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:42.853154 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a09fda-fb24-462f-8a5d-8fe6cff6bb93 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:42.895578 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2780c6c-6476-4fdc-bba8-dcac7eeb1133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:42.896346 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2780c6c-6476-4fdc-bba8-dcac7eeb1133 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:42.896684 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2780c6c-6476-4fdc-bba8-dcac7eeb1133 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:42.897003 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2780c6c-6476-4fdc-bba8-dcac7eeb1133 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:42.911568 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2780c6c-6476-4fdc-bba8-dcac7eeb1133 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:42.913334 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2780c6c-6476-4fdc-bba8-dcac7eeb1133 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:42.936443 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-e2780c6c-6476-4fdc-bba8-dcac7eeb1133 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:31:42.936443 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:31:42.936443 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:31:42.936443 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:31:42.936443 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:31:42.936443 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:31:42.936443 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:31:42.936443 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:31:42.936443 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:31:42.936443 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:31:42.936443 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:31:42.936443 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:31:42.936443 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:31:42.936443 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:31:42.936443 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:31:42.936443 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:31:42.937611 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:31:42.937611 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:31:42.937611 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:31:42.937611 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:31:42.937611 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:31:42.937611 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:31:42.937611 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:31:42.937611 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:31:42.937611 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:31:42.937611 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:31:42.937611 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:31:42.937611 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:31:42.937611 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:31:42.937611 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:31:42.937611 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:31:42.937611 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:31:42.937611 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:31:42.940491 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:31:42.940491 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:31:42.940491 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:31:42.943148 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 605/1203] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:31:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:31:42.944688 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd7cd4a-9409-4839-a7b9-979ccddee145 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:43.075410 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd7cd4a-9409-4839-a7b9-979ccddee145 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:43.118064 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:43.130237 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f575700a-a901-4dcb-ad84-1794130d07cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:43.131376 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f575700a-a901-4dcb-ad84-1794130d07cf None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:43.134227 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f575700a-a901-4dcb-ad84-1794130d07cf None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:43.134775 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f575700a-a901-4dcb-ad84-1794130d07cf None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:43.135474 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f575700a-a901-4dcb-ad84-1794130d07cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:31:43.135784 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f575700a-a901-4dcb-ad84-1794130d07cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:43.137662 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f575700a-a901-4dcb-ad84-1794130d07cf None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:43.150961 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 599/1204] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:31:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:43.158203 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2780c6c-6476-4fdc-bba8-dcac7eeb1133 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:43.236528 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:43.305812 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2780c6c-6476-4fdc-bba8-dcac7eeb1133 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:43.347742 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081e8fce-19fd-4579-8a1b-e46c64d9033d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:43.348416 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e8fce-19fd-4579-8a1b-e46c64d9033d None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:43.348646 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e8fce-19fd-4579-8a1b-e46c64d9033d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:43.348826 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e8fce-19fd-4579-8a1b-e46c64d9033d None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:43.349472 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081e8fce-19fd-4579-8a1b-e46c64d9033d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:43.351213 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081e8fce-19fd-4579-8a1b-e46c64d9033d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:43.386289 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 606/1205] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Sep 28 15:31:43 2022] POST /identity/v3/users => generated 251 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:31:45.704750 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e979d4-d523-4d8b-840f-ba67a3a28fd4 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:45.705362 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e979d4-d523-4d8b-840f-ba67a3a28fd4 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:45.705362 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e979d4-d523-4d8b-840f-ba67a3a28fd4 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:45.706686 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 600/1206] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:31:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:31:45.712543 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23195592-db20-4783-b841-88a7eb964431 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:45.714554 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23195592-db20-4783-b841-88a7eb964431 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:45.714554 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23195592-db20-4783-b841-88a7eb964431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:45.732945 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-23195592-db20-4783-b841-88a7eb964431 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:31:45.734386 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-23195592-db20-4783-b841-88a7eb964431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:31:45.817432 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 607/1207] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:31:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 28 15:31:46.153723 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e979d4-d523-4d8b-840f-ba67a3a28fd4 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:46.274869 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e979d4-d523-4d8b-840f-ba67a3a28fd4 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:46.324684 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f8f509-f759-4537-a1bc-90592583d9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:46.325320 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f8f509-f759-4537-a1bc-90592583d9b2 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:46.325604 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f8f509-f759-4537-a1bc-90592583d9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:46.325838 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f8f509-f759-4537-a1bc-90592583d9b2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:46.337163 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f8f509-f759-4537-a1bc-90592583d9b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:31:46.342471 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-c4f8f509-f759-4537-a1bc-90592583d9b2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:31:46.342471 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:31:46.342471 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:31:46.342471 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:31:46.342471 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:31:46.342471 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:31:46.342471 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:31:46.342471 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:31:46.342471 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:31:46.342471 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:31:46.342471 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:31:46.342471 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:31:46.342471 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:31:46.342471 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:31:46.342471 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:31:46.342471 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:31:46.343478 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:31:46.343478 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:31:46.343478 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:31:46.343478 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:31:46.343478 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:31:46.343478 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:31:46.343478 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:31:46.343478 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:31:46.343478 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:31:46.343478 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:31:46.343478 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:31:46.343478 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:31:46.343478 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:31:46.343478 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:31:46.343478 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:31:46.343478 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:31:46.346872 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:31:46.346872 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:31:46.346872 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:31:46.346872 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:31:46.346872 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:31:46.346872 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:31:46.346872 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:31:46.346872 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:31:46.346872 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:31:46.346872 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:31:46.346872 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:31:46.346872 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:31:46.346872 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 601/1208] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 15:31:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:31:46.349840 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23195592-db20-4783-b841-88a7eb964431 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:46.394151 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:46.458750 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23195592-db20-4783-b841-88a7eb964431 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:46.495091 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f70b80-d1e4-44e5-a369-6398c074cce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:46.495737 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f70b80-d1e4-44e5-a369-6398c074cce1 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:46.496003 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f70b80-d1e4-44e5-a369-6398c074cce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:46.496276 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f70b80-d1e4-44e5-a369-6398c074cce1 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:46.496945 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f70b80-d1e4-44e5-a369-6398c074cce1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:31:46.497299 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f70b80-d1e4-44e5-a369-6398c074cce1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:46.498631 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f70b80-d1e4-44e5-a369-6398c074cce1 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:46.512474 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 608/1209] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 15:31:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:46.513712 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f8f509-f759-4537-a1bc-90592583d9b2 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:46.569374 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:46.633915 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f8f509-f759-4537-a1bc-90592583d9b2 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:46.695709 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f233df-42d0-4ad8-a3a2-5b49bca060e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:46.697285 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f233df-42d0-4ad8-a3a2-5b49bca060e3 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:46.697555 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f233df-42d0-4ad8-a3a2-5b49bca060e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:46.697817 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f233df-42d0-4ad8-a3a2-5b49bca060e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:46.711287 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f233df-42d0-4ad8-a3a2-5b49bca060e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:46.713026 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f233df-42d0-4ad8-a3a2-5b49bca060e3 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:46.730190 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-07f233df-42d0-4ad8-a3a2-5b49bca060e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:31:46.730190 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:31:46.730190 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:31:46.730190 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:31:46.730190 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:31:46.730190 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:31:46.730190 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:31:46.730190 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:31:46.730190 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:31:46.730190 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:31:46.730190 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:31:46.730190 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:31:46.730190 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:31:46.730190 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:31:46.730190 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:31:46.730190 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:31:46.731454 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:31:46.731454 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:31:46.731454 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:31:46.731454 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:31:46.731454 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:31:46.731454 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:31:46.731454 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:31:46.731454 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:31:46.731454 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:31:46.731454 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:31:46.731454 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:31:46.731454 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:31:46.731454 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:31:46.731454 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:31:46.731454 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:31:46.731454 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:31:46.731454 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:31:46.732578 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:31:46.732578 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:31:46.732578 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:31:46.737485 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 602/1210] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:31:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:31:46.738567 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f70b80-d1e4-44e5-a369-6398c074cce1 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:46.847273 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:46.850508 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f70b80-d1e4-44e5-a369-6398c074cce1 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:46.890925 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d184ed-2658-4326-9444-f50d3aa06899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:46.891666 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d184ed-2658-4326-9444-f50d3aa06899 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:46.891964 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d184ed-2658-4326-9444-f50d3aa06899 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:46.892273 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d184ed-2658-4326-9444-f50d3aa06899 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:46.893050 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d184ed-2658-4326-9444-f50d3aa06899 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:31:46.893397 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d184ed-2658-4326-9444-f50d3aa06899 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:46.895192 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d184ed-2658-4326-9444-f50d3aa06899 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:46.911193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 609/1211] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:31:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:46.917581 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f233df-42d0-4ad8-a3a2-5b49bca060e3 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:47.034399 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f233df-42d0-4ad8-a3a2-5b49bca060e3 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:47.036249 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:47.072799 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc85a2fe-dbf2-4503-adcf-1fb85ac9e62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:47.073484 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc85a2fe-dbf2-4503-adcf-1fb85ac9e62b None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:47.074326 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc85a2fe-dbf2-4503-adcf-1fb85ac9e62b None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:47.074600 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc85a2fe-dbf2-4503-adcf-1fb85ac9e62b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:47.082066 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc85a2fe-dbf2-4503-adcf-1fb85ac9e62b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:47.083506 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc85a2fe-dbf2-4503-adcf-1fb85ac9e62b None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:47.095494 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-bc85a2fe-dbf2-4503-adcf-1fb85ac9e62b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 28 15:31:47.095494 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:31:47.095494 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:31:47.095494 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:31:47.095494 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:31:47.095494 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:31:47.095494 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:31:47.095494 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:31:47.095494 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:31:47.095494 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:31:47.095494 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:31:47.095494 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:31:47.095494 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:31:47.095494 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:31:47.095494 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:31:47.095494 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:31:47.096503 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:31:47.096503 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:31:47.096503 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:31:47.096503 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:31:47.096503 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:31:47.096503 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:31:47.096503 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:31:47.096503 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:31:47.096503 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:31:47.096503 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:31:47.096503 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:31:47.096503 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:31:47.096503 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:31:47.096503 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:31:47.096503 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:31:47.096503 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:31:47.096503 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:31:47.096503 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:31:47.097522 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:31:47.097522 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:31:47.097522 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:31:47.097522 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:31:47.097522 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:31:47.097522 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:31:47.097522 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:31:47.097522 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:31:47.097522 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 28 15:31:47.097522 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:31:47.097522 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 603/1212] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:31:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:31:47.104614 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d184ed-2658-4326-9444-f50d3aa06899 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:47.218770 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d184ed-2658-4326-9444-f50d3aa06899 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:47.264513 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da359e3-7cda-4a0f-9ede-51d69b327082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:47.265125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da359e3-7cda-4a0f-9ede-51d69b327082 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:47.265401 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da359e3-7cda-4a0f-9ede-51d69b327082 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:47.265670 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da359e3-7cda-4a0f-9ede-51d69b327082 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:47.266422 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da359e3-7cda-4a0f-9ede-51d69b327082 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:31:47.266775 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da359e3-7cda-4a0f-9ede-51d69b327082 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:47.268193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da359e3-7cda-4a0f-9ede-51d69b327082 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:47.298006 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 610/1213] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Sep 28 15:31:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:47.300984 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc85a2fe-dbf2-4503-adcf-1fb85ac9e62b None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:47.426078 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc85a2fe-dbf2-4503-adcf-1fb85ac9e62b None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:47.465906 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b2b952-c76b-4d32-a202-c0f02e6e5ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:47.466664 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b2b952-c76b-4d32-a202-c0f02e6e5ba5 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:47.467057 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b2b952-c76b-4d32-a202-c0f02e6e5ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:47.467424 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b2b952-c76b-4d32-a202-c0f02e6e5ba5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:47.505940 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:47.551552 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b2b952-c76b-4d32-a202-c0f02e6e5ba5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:47.553139 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b2b952-c76b-4d32-a202-c0f02e6e5ba5 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:47.679553 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 604/1214] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:31:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 15:31:47.680894 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da359e3-7cda-4a0f-9ede-51d69b327082 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:47.703543 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:47.803148 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da359e3-7cda-4a0f-9ede-51d69b327082 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:47.841139 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e758f0-9ff9-4438-983a-ce572245ab6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:47.845352 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e758f0-9ff9-4438-983a-ce572245ab6d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:47.845352 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e758f0-9ff9-4438-983a-ce572245ab6d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:47.845352 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e758f0-9ff9-4438-983a-ce572245ab6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:47.856922 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e758f0-9ff9-4438-983a-ce572245ab6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:47.858422 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e758f0-9ff9-4438-983a-ce572245ab6d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:47.877969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-27e758f0-9ff9-4438-983a-ce572245ab6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:31:47.877969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:31:47.877969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:31:47.877969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:31:47.877969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:31:47.877969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:31:47.877969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:31:47.877969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:31:47.877969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:31:47.877969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:31:47.877969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:31:47.877969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:31:47.877969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:31:47.877969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:31:47.877969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:31:47.877969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:31:47.879273 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:31:47.879273 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:31:47.879273 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:31:47.879273 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:31:47.879273 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:31:47.879273 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:31:47.879273 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:31:47.879273 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:31:47.879273 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:31:47.879273 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:31:47.879273 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:31:47.879273 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:31:47.879273 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:31:47.879273 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:31:47.879273 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:31:47.879273 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:31:47.879273 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:31:47.880354 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:31:47.880354 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:31:47.880354 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:31:47.880635 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 611/1215] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:31:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:31:47.886983 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b2b952-c76b-4d32-a202-c0f02e6e5ba5 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:47.952558 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:48.000967 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b2b952-c76b-4d32-a202-c0f02e6e5ba5 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:48.040969 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6234d0f-b757-4cba-b88d-1494360b43ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:48.041992 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6234d0f-b757-4cba-b88d-1494360b43ed None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:48.042986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6234d0f-b757-4cba-b88d-1494360b43ed None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:48.043609 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6234d0f-b757-4cba-b88d-1494360b43ed None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:48.044360 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6234d0f-b757-4cba-b88d-1494360b43ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:31:48.044661 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6234d0f-b757-4cba-b88d-1494360b43ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:48.046249 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6234d0f-b757-4cba-b88d-1494360b43ed None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:48.060203 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 605/1216] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:31:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:48.065920 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e758f0-9ff9-4438-983a-ce572245ab6d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:48.140000 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:48.173510 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e758f0-9ff9-4438-983a-ce572245ab6d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:48.208595 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eec0ba-5713-41cb-bf68-6c145d321ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:48.209218 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eec0ba-5713-41cb-bf68-6c145d321ff8 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:48.209499 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eec0ba-5713-41cb-bf68-6c145d321ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:48.209772 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eec0ba-5713-41cb-bf68-6c145d321ff8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:48.216501 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eec0ba-5713-41cb-bf68-6c145d321ff8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:48.217984 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eec0ba-5713-41cb-bf68-6c145d321ff8 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:48.229217 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-d7eec0ba-5713-41cb-bf68-6c145d321ff8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:31:48.229217 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:31:48.229217 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:31:48.229217 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:31:48.229217 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:31:48.229217 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:31:48.229217 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:31:48.229217 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:31:48.229217 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:31:48.229217 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:31:48.229217 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:31:48.229217 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:31:48.229217 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:31:48.229217 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:31:48.229217 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:31:48.230928 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:31:48.230928 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:31:48.230928 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:31:48.230928 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:31:48.230928 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:31:48.230928 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:31:48.230928 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:31:48.230928 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:31:48.230928 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:31:48.230928 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:31:48.230928 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:31:48.230928 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:31:48.230928 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:31:48.230928 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:31:48.230928 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:31:48.230928 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:31:48.230928 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:31:48.232001 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:31:48.232001 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:31:48.232001 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:31:48.232001 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:31:48.232001 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:31:48.232001 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:31:48.232001 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:31:48.232001 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:31:48.235858 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 612/1217] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:31:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:31:48.236754 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6234d0f-b757-4cba-b88d-1494360b43ed None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:48.352383 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6234d0f-b757-4cba-b88d-1494360b43ed None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:48.375469 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:48.387802 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae3f007-3a2c-4d78-bdb1-c13351fb8725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:48.388473 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3f007-3a2c-4d78-bdb1-c13351fb8725 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:48.388780 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3f007-3a2c-4d78-bdb1-c13351fb8725 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:48.389061 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3f007-3a2c-4d78-bdb1-c13351fb8725 None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:48.389835 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae3f007-3a2c-4d78-bdb1-c13351fb8725 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:31:48.390167 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae3f007-3a2c-4d78-bdb1-c13351fb8725 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:48.391951 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae3f007-3a2c-4d78-bdb1-c13351fb8725 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:48.406859 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 606/1218] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 15:31:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:48.414122 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eec0ba-5713-41cb-bf68-6c145d321ff8 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:48.529641 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eec0ba-5713-41cb-bf68-6c145d321ff8 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:48.560530 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:48.567037 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98504a0a-4d37-46aa-8e9b-e7a729f26dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:48.567633 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98504a0a-4d37-46aa-8e9b-e7a729f26dec None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:48.567935 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98504a0a-4d37-46aa-8e9b-e7a729f26dec None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:48.568236 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98504a0a-4d37-46aa-8e9b-e7a729f26dec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:48.569094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98504a0a-4d37-46aa-8e9b-e7a729f26dec None admin] RBAC: Adding query filter params (role.id=342b64fe0cdb4925b23c6cd461a152ac, scope.project.id=3dc6c5f5fcd34922b0d09cd2aaed6cf1, user.id=de934d0105bd4d01ba6c06a4a0d36704) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:31:48.569428 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98504a0a-4d37-46aa-8e9b-e7a729f26dec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:48.570951 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98504a0a-4d37-46aa-8e9b-e7a729f26dec None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:48.576665 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 613/1219] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 15:31:48 2022] GET /identity/v3/role_assignments?user.id=de934d0105bd4d01ba6c06a4a0d36704&scope.project.id=3dc6c5f5fcd34922b0d09cd2aaed6cf1&role.id=342b64fe0cdb4925b23c6cd461a152ac => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:50.859618 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d4c45-106f-49fe-ba39-782203931d5a None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:50.860262 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d4c45-106f-49fe-ba39-782203931d5a None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:50.860262 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d4c45-106f-49fe-ba39-782203931d5a None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:50.861602 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 607/1220] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:31:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:31:50.867093 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81be05-1f80-455b-bcd1-e527863d213f None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:50.867565 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81be05-1f80-455b-bcd1-e527863d213f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:50.867829 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81be05-1f80-455b-bcd1-e527863d213f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:50.889437 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-2d81be05-1f80-455b-bcd1-e527863d213f None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:31:50.890661 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2d81be05-1f80-455b-bcd1-e527863d213f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:31:50.969816 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 614/1221] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:31:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:31:51.248049 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d4c45-106f-49fe-ba39-782203931d5a None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:51.366100 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d4c45-106f-49fe-ba39-782203931d5a None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:51.413885 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aad2ae-c171-4183-a2dc-c8bbde6a05f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:51.414528 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aad2ae-c171-4183-a2dc-c8bbde6a05f5 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:51.414858 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aad2ae-c171-4183-a2dc-c8bbde6a05f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:51.415102 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aad2ae-c171-4183-a2dc-c8bbde6a05f5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:51.426958 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aad2ae-c171-4183-a2dc-c8bbde6a05f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:31:51.431785 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-f4aad2ae-c171-4183-a2dc-c8bbde6a05f5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:31:51.431785 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:31:51.431785 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:31:51.431785 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:31:51.431785 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:31:51.431785 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:31:51.431785 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:31:51.431785 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:31:51.431785 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:31:51.431785 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:31:51.431785 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:31:51.431785 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:31:51.431785 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:31:51.431785 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:31:51.431785 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:31:51.431785 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:31:51.438054 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:31:51.438054 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:31:51.438054 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:31:51.438054 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:31:51.438054 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:31:51.438054 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:31:51.438054 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:31:51.438054 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:31:51.438054 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:31:51.438054 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:31:51.438054 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:31:51.438054 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:31:51.438054 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:31:51.438054 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:31:51.438054 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:31:51.438054 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:31:51.439222 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:31:51.439222 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:31:51.439222 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:31:51.439222 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:31:51.439222 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:31:51.439222 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:31:51.439222 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:31:51.439222 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:31:51.439222 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:31:51.439222 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:31:51.439222 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:31:51.439222 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:31:51.439222 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 608/1222] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 15:31:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:31:51.440315 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81be05-1f80-455b-bcd1-e527863d213f None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:51.572302 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81be05-1f80-455b-bcd1-e527863d213f None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:51.599283 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:51.612005 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8c6e89-b490-48fd-9b25-f5ec52088e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:51.612781 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c6e89-b490-48fd-9b25-f5ec52088e9f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:51.613064 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c6e89-b490-48fd-9b25-f5ec52088e9f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:51.613428 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c6e89-b490-48fd-9b25-f5ec52088e9f None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:51.614223 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8c6e89-b490-48fd-9b25-f5ec52088e9f None admin] RBAC: Adding query filter params (name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:31:51.614551 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8c6e89-b490-48fd-9b25-f5ec52088e9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:51.616144 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8c6e89-b490-48fd-9b25-f5ec52088e9f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:51.624746 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 615/1223] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 15:31:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:51.631340 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aad2ae-c171-4183-a2dc-c8bbde6a05f5 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:51.680634 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:51.748339 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aad2ae-c171-4183-a2dc-c8bbde6a05f5 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:51.788473 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30316e06-502a-4f9e-b252-615da8ce904c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:51.789090 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30316e06-502a-4f9e-b252-615da8ce904c None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:51.789361 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30316e06-502a-4f9e-b252-615da8ce904c None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:51.789625 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30316e06-502a-4f9e-b252-615da8ce904c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:51.799868 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30316e06-502a-4f9e-b252-615da8ce904c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:51.801304 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30316e06-502a-4f9e-b252-615da8ce904c None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:51.815549 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-30316e06-502a-4f9e-b252-615da8ce904c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:31:51.815549 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:31:51.815549 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:31:51.815549 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:31:51.815549 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:31:51.815549 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:31:51.815549 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:31:51.815549 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:31:51.815549 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:31:51.815549 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:31:51.815549 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:31:51.815549 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:31:51.815549 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:31:51.815549 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:31:51.815549 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:31:51.815549 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:31:51.816561 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:31:51.816561 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:31:51.816561 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:31:51.816561 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:31:51.816561 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:31:51.816561 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:31:51.816561 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:31:51.816561 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:31:51.816561 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:31:51.816561 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:31:51.816561 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:31:51.816561 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:31:51.816561 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:31:51.816561 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:31:51.816561 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:31:51.816561 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:31:51.816561 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:31:51.817536 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:31:51.817536 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:31:51.817536 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:31:51.817536 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 609/1224] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:31:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:31:51.822665 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8c6e89-b490-48fd-9b25-f5ec52088e9f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:51.934875 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8c6e89-b490-48fd-9b25-f5ec52088e9f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:51.970045 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c428e020-5efd-4bcc-8274-f323ced62abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:51.970655 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428e020-5efd-4bcc-8274-f323ced62abc None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:51.970979 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428e020-5efd-4bcc-8274-f323ced62abc None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:51.971284 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428e020-5efd-4bcc-8274-f323ced62abc None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:51.972001 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c428e020-5efd-4bcc-8274-f323ced62abc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:31:51.972337 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c428e020-5efd-4bcc-8274-f323ced62abc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:51.973898 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c428e020-5efd-4bcc-8274-f323ced62abc None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:51.987220 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 616/1225] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:31:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:51.993352 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30316e06-502a-4f9e-b252-615da8ce904c None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:52.102640 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:52.111157 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30316e06-502a-4f9e-b252-615da8ce904c None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:52.152750 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8fcad2-ed2e-4d5c-8d20-128daeb02ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:52.153520 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8fcad2-ed2e-4d5c-8d20-128daeb02ea0 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:52.153934 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8fcad2-ed2e-4d5c-8d20-128daeb02ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:52.154207 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8fcad2-ed2e-4d5c-8d20-128daeb02ea0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:52.163207 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8fcad2-ed2e-4d5c-8d20-128daeb02ea0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:52.165086 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8fcad2-ed2e-4d5c-8d20-128daeb02ea0 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:52.176801 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-3c8fcad2-ed2e-4d5c-8d20-128daeb02ea0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 28 15:31:52.176801 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:31:52.176801 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:31:52.176801 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:31:52.176801 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:31:52.176801 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:31:52.176801 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:31:52.176801 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:31:52.176801 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:31:52.176801 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:31:52.176801 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:31:52.176801 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:31:52.176801 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:31:52.176801 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:31:52.176801 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:31:52.176801 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:31:52.177920 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:31:52.177920 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:31:52.177920 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:31:52.177920 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:31:52.177920 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:31:52.177920 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:31:52.177920 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:31:52.177920 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:31:52.177920 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:31:52.177920 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:31:52.177920 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:31:52.177920 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:31:52.177920 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:31:52.177920 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:31:52.177920 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:31:52.177920 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:31:52.177920 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:31:52.177920 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:31:52.179320 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:31:52.179320 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:31:52.179320 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:31:52.179320 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:31:52.179320 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:31:52.179320 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:31:52.179320 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:31:52.179320 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:31:52.179320 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 28 15:31:52.179320 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:31:52.179320 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 610/1226] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:31:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:31:52.184129 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c428e020-5efd-4bcc-8274-f323ced62abc None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:52.234362 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:52.293505 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c428e020-5efd-4bcc-8274-f323ced62abc None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:52.329738 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76420d12-e074-4380-9563-818ebc3201cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:52.330428 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76420d12-e074-4380-9563-818ebc3201cf None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:52.330818 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76420d12-e074-4380-9563-818ebc3201cf None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:52.331150 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76420d12-e074-4380-9563-818ebc3201cf None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:52.331894 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76420d12-e074-4380-9563-818ebc3201cf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:31:52.332235 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76420d12-e074-4380-9563-818ebc3201cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:52.333782 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76420d12-e074-4380-9563-818ebc3201cf None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:52.354444 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 617/1227] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Sep 28 15:31:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:52.360891 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8fcad2-ed2e-4d5c-8d20-128daeb02ea0 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:52.471597 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8fcad2-ed2e-4d5c-8d20-128daeb02ea0 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:52.509734 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff10ca5-85a5-4bb6-9622-047beffce79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:52.510456 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff10ca5-85a5-4bb6-9622-047beffce79f None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:52.511218 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff10ca5-85a5-4bb6-9622-047beffce79f None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:52.511523 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff10ca5-85a5-4bb6-9622-047beffce79f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:52.524044 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff10ca5-85a5-4bb6-9622-047beffce79f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:52.525483 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff10ca5-85a5-4bb6-9622-047beffce79f None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:52.543056 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-6ff10ca5-85a5-4bb6-9622-047beffce79f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:31:52.543056 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:31:52.543056 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:31:52.543056 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:31:52.543056 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:31:52.543056 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:31:52.543056 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:31:52.543056 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:31:52.543056 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:31:52.543056 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:31:52.543056 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:31:52.543056 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:31:52.543056 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:31:52.543056 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:31:52.543056 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:31:52.543056 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:31:52.544643 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:31:52.544643 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:31:52.544643 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:31:52.544643 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:31:52.544643 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:31:52.544643 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:31:52.544643 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:31:52.544643 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:31:52.544643 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:31:52.544643 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:31:52.544643 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:31:52.544643 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:31:52.544643 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:31:52.544643 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:31:52.544643 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:31:52.544643 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:31:52.544643 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:31:52.545755 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:31:52.545755 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:31:52.545755 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:31:52.545755 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 611/1228] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:31:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:31:52.550420 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76420d12-e074-4380-9563-818ebc3201cf None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:52.665050 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76420d12-e074-4380-9563-818ebc3201cf None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:52.667797 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:52.702245 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27853ea0-dbe5-4f51-8e6c-b68f52f9b098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:52.702852 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27853ea0-dbe5-4f51-8e6c-b68f52f9b098 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:52.703131 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27853ea0-dbe5-4f51-8e6c-b68f52f9b098 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:52.703404 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27853ea0-dbe5-4f51-8e6c-b68f52f9b098 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:52.704034 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27853ea0-dbe5-4f51-8e6c-b68f52f9b098 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:31:52.704335 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27853ea0-dbe5-4f51-8e6c-b68f52f9b098 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:52.705696 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27853ea0-dbe5-4f51-8e6c-b68f52f9b098 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:52.717364 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 618/1229] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:31:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:31:52.723627 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff10ca5-85a5-4bb6-9622-047beffce79f None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:52.733300 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:52.827009 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff10ca5-85a5-4bb6-9622-047beffce79f None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:52.860344 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02485a36-5f67-4e9c-82f6-8d09888c7c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:52.860973 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02485a36-5f67-4e9c-82f6-8d09888c7c4d None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:52.861246 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02485a36-5f67-4e9c-82f6-8d09888c7c4d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:52.861511 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02485a36-5f67-4e9c-82f6-8d09888c7c4d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:52.868341 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02485a36-5f67-4e9c-82f6-8d09888c7c4d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:52.869753 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02485a36-5f67-4e9c-82f6-8d09888c7c4d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:52.883439 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-02485a36-5f67-4e9c-82f6-8d09888c7c4d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:31:52.883439 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:31:52.883439 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:31:52.883439 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:31:52.883439 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:31:52.883439 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:31:52.883439 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:31:52.883439 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:31:52.883439 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:31:52.883439 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:31:52.883439 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:31:52.883439 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:31:52.883439 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:31:52.883439 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:31:52.883439 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:31:52.884627 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:31:52.884627 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:31:52.884627 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:31:52.884627 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:31:52.884627 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:31:52.884627 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:31:52.884627 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:31:52.884627 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:31:52.884627 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:31:52.884627 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:31:52.884627 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:31:52.884627 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:31:52.884627 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:31:52.884627 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:31:52.884627 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:31:52.884627 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:31:52.884627 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:31:52.885686 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:31:52.885686 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:31:52.885686 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:31:52.885686 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:31:52.885686 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:31:52.885686 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:31:52.885686 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:31:52.885686 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:31:52.885686 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 612/1230] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:31:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:31:52.891165 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27853ea0-dbe5-4f51-8e6c-b68f52f9b098 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:53.004798 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27853ea0-dbe5-4f51-8e6c-b68f52f9b098 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:53.047938 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b849073-6cf4-4f25-8e6d-c36916c96ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:53.048715 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b849073-6cf4-4f25-8e6d-c36916c96ec3 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:53.049101 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b849073-6cf4-4f25-8e6d-c36916c96ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:53.049394 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b849073-6cf4-4f25-8e6d-c36916c96ec3 None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:53.050234 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b849073-6cf4-4f25-8e6d-c36916c96ec3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:31:53.050616 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b849073-6cf4-4f25-8e6d-c36916c96ec3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:53.052239 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b849073-6cf4-4f25-8e6d-c36916c96ec3 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:53.066490 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 619/1231] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 15:31:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:53.072581 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02485a36-5f67-4e9c-82f6-8d09888c7c4d None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:53.149530 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:53.187767 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02485a36-5f67-4e9c-82f6-8d09888c7c4d None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:53.223375 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca1430d-8220-4ddb-aac5-a343c653b114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:53.224053 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1430d-8220-4ddb-aac5-a343c653b114 None admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:53.224332 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1430d-8220-4ddb-aac5-a343c653b114 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:53.224606 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1430d-8220-4ddb-aac5-a343c653b114 None admin] PATH_INFO: `/v3/projects/3dc6c5f5fcd34922b0d09cd2aaed6cf1/users/de934d0105bd4d01ba6c06a4a0d36704/roles/342b64fe0cdb4925b23c6cd461a152ac` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:53.239431 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca1430d-8220-4ddb-aac5-a343c653b114 None admin] RBAC: Authorizing `identity:create_grant(project_id=3dc6c5f5fcd34922b0d09cd2aaed6cf1, user_id=de934d0105bd4d01ba6c06a4a0d36704, role_id=342b64fe0cdb4925b23c6cd461a152ac)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:53.240857 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca1430d-8220-4ddb-aac5-a343c653b114 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:53.253233 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 613/1232] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 15:31:53 2022] PUT /identity/v3/projects/3dc6c5f5fcd34922b0d09cd2aaed6cf1/users/de934d0105bd4d01ba6c06a4a0d36704/roles/342b64fe0cdb4925b23c6cd461a152ac => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:31:55.425906 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa072b-ecf7-46de-8db9-4615c81ddc35 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:55.426609 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa072b-ecf7-46de-8db9-4615c81ddc35 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:55.426609 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa072b-ecf7-46de-8db9-4615c81ddc35 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:55.428379 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 620/1233] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:31:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:31:55.434172 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468cae5b-da3f-42b2-8739-51453ac209d1 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:55.434630 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468cae5b-da3f-42b2-8739-51453ac209d1 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:55.434736 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468cae5b-da3f-42b2-8739-51453ac209d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:55.453332 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-468cae5b-da3f-42b2-8739-51453ac209d1 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:31:55.454539 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-468cae5b-da3f-42b2-8739-51453ac209d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:31:55.533233 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 614/1234] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:31:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:31:55.835732 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aa072b-ecf7-46de-8db9-4615c81ddc35 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:55.860140 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:55.948571 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aa072b-ecf7-46de-8db9-4615c81ddc35 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:55.988076 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd3bd6-b378-4f09-8271-2613357ffcb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:55.988829 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd3bd6-b378-4f09-8271-2613357ffcb4 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:55.989452 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd3bd6-b378-4f09-8271-2613357ffcb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:55.989452 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd3bd6-b378-4f09-8271-2613357ffcb4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:56.000999 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfd3bd6-b378-4f09-8271-2613357ffcb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:31:56.006668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-ccfd3bd6-b378-4f09-8271-2613357ffcb4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:31:56.006668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:31:56.006668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:31:56.006668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:31:56.006668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:31:56.006668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:31:56.006668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:31:56.006668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:31:56.006668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:31:56.006668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:31:56.006668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:31:56.006668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:31:56.006668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:31:56.006668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:31:56.006668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:31:56.006668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:31:56.008094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:31:56.008094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:31:56.008094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:31:56.008094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:31:56.008094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:31:56.008094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:31:56.008094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:31:56.008094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:31:56.008094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:31:56.008094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:31:56.008094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:31:56.008094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:31:56.008094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:31:56.008094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:31:56.008094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:31:56.008094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:31:56.009031 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:31:56.009031 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:31:56.009031 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:31:56.009031 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:31:56.009031 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:31:56.009031 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:31:56.009031 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:31:56.009031 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:31:56.009031 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:31:56.009031 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:31:56.009031 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 28 15:31:56.009031 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:31:56.009031 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 621/1235] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Sep 28 15:31:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:31:56.014878 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468cae5b-da3f-42b2-8739-51453ac209d1 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:56.072216 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:56.132968 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468cae5b-da3f-42b2-8739-51453ac209d1 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:56.172454 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bca4da-1c21-46de-a465-3e9691e6b86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:56.173169 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bca4da-1c21-46de-a465-3e9691e6b86e None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:56.173465 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bca4da-1c21-46de-a465-3e9691e6b86e None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:56.173770 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bca4da-1c21-46de-a465-3e9691e6b86e None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:56.174518 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bca4da-1c21-46de-a465-3e9691e6b86e None admin] RBAC: Adding query filter params (name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:31:56.174794 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bca4da-1c21-46de-a465-3e9691e6b86e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:56.176457 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bca4da-1c21-46de-a465-3e9691e6b86e None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:56.185148 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 615/1236] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Sep 28 15:31:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:56.191436 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd3bd6-b378-4f09-8271-2613357ffcb4 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:56.281274 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:56.301343 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd3bd6-b378-4f09-8271-2613357ffcb4 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:56.338089 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52037a7b-e012-4e17-af96-2880db9a2584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:56.339483 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52037a7b-e012-4e17-af96-2880db9a2584 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:56.339801 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52037a7b-e012-4e17-af96-2880db9a2584 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:56.340111 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52037a7b-e012-4e17-af96-2880db9a2584 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:56.350618 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52037a7b-e012-4e17-af96-2880db9a2584 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:56.352488 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52037a7b-e012-4e17-af96-2880db9a2584 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:56.371541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-52037a7b-e012-4e17-af96-2880db9a2584 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:31:56.371541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:31:56.371541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:31:56.371541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:31:56.371541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:31:56.371541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:31:56.371541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:31:56.371541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:31:56.371541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:31:56.371541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:31:56.371541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:31:56.371541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:31:56.371541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:31:56.371541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:31:56.371541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:31:56.371541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:31:56.373541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:31:56.373541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:31:56.373541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:31:56.373541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:31:56.373541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:31:56.373541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:31:56.373541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:31:56.373541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:31:56.373541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:31:56.373541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:31:56.373541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:31:56.373541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:31:56.373541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:31:56.373541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:31:56.373541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:31:56.373541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:31:56.373541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:31:56.374936 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:31:56.374936 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:31:56.374936 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:31:56.374936 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 622/1237] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:31:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:31:56.379090 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bca4da-1c21-46de-a465-3e9691e6b86e None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:56.491003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bca4da-1c21-46de-a465-3e9691e6b86e None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:56.501333 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:56.530489 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd705767-68e1-459a-9964-d5570d750d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:56.531201 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd705767-68e1-459a-9964-d5570d750d2f None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:56.531529 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd705767-68e1-459a-9964-d5570d750d2f None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:56.531798 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd705767-68e1-459a-9964-d5570d750d2f None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:56.532438 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd705767-68e1-459a-9964-d5570d750d2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:31:56.532733 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd705767-68e1-459a-9964-d5570d750d2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:56.534099 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd705767-68e1-459a-9964-d5570d750d2f None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:56.546012 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 616/1238] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:31:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:56.551892 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52037a7b-e012-4e17-af96-2880db9a2584 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:56.660624 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52037a7b-e012-4e17-af96-2880db9a2584 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:56.694795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe2d5a0-e096-47a1-8ff9-0b806607f2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:56.695513 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2d5a0-e096-47a1-8ff9-0b806607f2dc None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:56.695856 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2d5a0-e096-47a1-8ff9-0b806607f2dc None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:56.696190 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2d5a0-e096-47a1-8ff9-0b806607f2dc None admin] PATH_INFO: `/v3/users/placement` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:56.699637 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:56.705403 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe2d5a0-e096-47a1-8ff9-0b806607f2dc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:56.707592 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe2d5a0-e096-47a1-8ff9-0b806607f2dc None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:56.720932 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-0fe2d5a0-e096-47a1-8ff9-0b806607f2dc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 28 15:31:56.720932 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:31:56.720932 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:31:56.720932 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:31:56.720932 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:31:56.720932 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:31:56.720932 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:31:56.720932 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:31:56.720932 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:31:56.720932 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:31:56.720932 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:31:56.720932 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:31:56.720932 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:31:56.720932 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:31:56.720932 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:31:56.720932 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:31:56.721935 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:31:56.721935 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:31:56.721935 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:31:56.721935 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:31:56.721935 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:31:56.721935 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:31:56.721935 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:31:56.721935 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:31:56.721935 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:31:56.721935 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:31:56.721935 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:31:56.721935 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:31:56.721935 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:31:56.721935 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:31:56.721935 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:31:56.721935 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:31:56.721935 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:31:56.721935 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:31:56.722909 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:31:56.722909 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:31:56.722909 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:31:56.722909 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:31:56.722909 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:31:56.722909 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:31:56.722909 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:31:56.722909 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:31:56.722909 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 28 15:31:56.722909 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:31:56.722909 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 623/1239] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:31:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:31:56.728436 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd705767-68e1-459a-9964-d5570d750d2f None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:56.844318 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd705767-68e1-459a-9964-d5570d750d2f None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:56.882148 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43365ef-fe9a-47bd-942c-bd5066264872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:56.882821 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43365ef-fe9a-47bd-942c-bd5066264872 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:56.883108 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43365ef-fe9a-47bd-942c-bd5066264872 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:56.883391 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43365ef-fe9a-47bd-942c-bd5066264872 None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:56.884278 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43365ef-fe9a-47bd-942c-bd5066264872 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:31:56.884579 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43365ef-fe9a-47bd-942c-bd5066264872 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:56.886919 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43365ef-fe9a-47bd-942c-bd5066264872 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:56.906254 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 617/1240] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Sep 28 15:31:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:31:56.912649 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe2d5a0-e096-47a1-8ff9-0b806607f2dc None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:57.028027 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe2d5a0-e096-47a1-8ff9-0b806607f2dc None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:57.063386 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d805624-ad20-4a4f-bbec-52d9624b8564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:57.063987 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d805624-ad20-4a4f-bbec-52d9624b8564 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:57.064268 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d805624-ad20-4a4f-bbec-52d9624b8564 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:57.064539 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d805624-ad20-4a4f-bbec-52d9624b8564 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:57.136172 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:57.143285 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d805624-ad20-4a4f-bbec-52d9624b8564 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:57.144850 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d805624-ad20-4a4f-bbec-52d9624b8564 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:57.251232 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 624/1241] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:31:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 15:31:57.257138 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43365ef-fe9a-47bd-942c-bd5066264872 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:57.287709 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:57.380695 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43365ef-fe9a-47bd-942c-bd5066264872 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:57.423148 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88daa3ca-178e-4569-9340-76e7e60ac18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:57.423703 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88daa3ca-178e-4569-9340-76e7e60ac18f None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:57.424075 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88daa3ca-178e-4569-9340-76e7e60ac18f None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:57.424350 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88daa3ca-178e-4569-9340-76e7e60ac18f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:57.436923 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88daa3ca-178e-4569-9340-76e7e60ac18f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:57.438484 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88daa3ca-178e-4569-9340-76e7e60ac18f None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:57.455158 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-88daa3ca-178e-4569-9340-76e7e60ac18f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:31:57.455158 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:31:57.455158 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:31:57.455158 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:31:57.455158 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:31:57.455158 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:31:57.455158 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:31:57.455158 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:31:57.455158 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:31:57.455158 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:31:57.455158 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:31:57.455158 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:31:57.455158 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:31:57.455158 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:31:57.455158 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:31:57.455158 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:31:57.456307 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:31:57.456307 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:31:57.456307 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:31:57.456307 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:31:57.456307 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:31:57.456307 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:31:57.456307 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:31:57.456307 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:31:57.456307 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:31:57.456307 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:31:57.456307 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:31:57.456307 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:31:57.456307 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:31:57.456307 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:31:57.456307 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:31:57.456307 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:31:57.456307 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:31:57.457278 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:31:57.457278 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:31:57.457278 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:31:57.457278 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 618/1242] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:31:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:31:57.462320 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d805624-ad20-4a4f-bbec-52d9624b8564 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:57.565871 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:57.573976 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d805624-ad20-4a4f-bbec-52d9624b8564 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:57.611833 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabf5a2d-8046-4857-80f4-62d826d4e08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:57.612439 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf5a2d-8046-4857-80f4-62d826d4e08f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:57.612762 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf5a2d-8046-4857-80f4-62d826d4e08f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:57.613080 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf5a2d-8046-4857-80f4-62d826d4e08f None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:57.613717 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabf5a2d-8046-4857-80f4-62d826d4e08f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:31:57.614048 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabf5a2d-8046-4857-80f4-62d826d4e08f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:57.615498 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabf5a2d-8046-4857-80f4-62d826d4e08f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:57.628263 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 625/1243] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:31:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:57.634380 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88daa3ca-178e-4569-9340-76e7e60ac18f None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:57.722208 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:31:57.752616 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88daa3ca-178e-4569-9340-76e7e60ac18f None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:57.795371 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea46136f-e853-4f73-9855-281f48987d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:57.795995 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46136f-e853-4f73-9855-281f48987d07 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:57.796272 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46136f-e853-4f73-9855-281f48987d07 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:57.796541 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46136f-e853-4f73-9855-281f48987d07 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:57.803863 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea46136f-e853-4f73-9855-281f48987d07 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:57.805295 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea46136f-e853-4f73-9855-281f48987d07 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:57.816610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-ea46136f-e853-4f73-9855-281f48987d07 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:31:57.816610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:31:57.816610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:31:57.816610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:31:57.816610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:31:57.816610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:31:57.816610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:31:57.816610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:31:57.816610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:31:57.816610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:31:57.816610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:31:57.816610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:31:57.816610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:31:57.816610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:31:57.816610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:31:57.817971 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:31:57.817971 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:31:57.817971 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:31:57.817971 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:31:57.817971 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:31:57.817971 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:31:57.817971 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:31:57.817971 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:31:57.817971 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:31:57.817971 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:31:57.817971 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:31:57.817971 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:31:57.817971 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:31:57.817971 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:31:57.817971 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:31:57.817971 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:31:57.817971 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:31:57.818941 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:31:57.818941 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:31:57.818941 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:31:57.818941 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:31:57.818941 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:31:57.818941 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:31:57.818941 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:31:57.818941 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:31:57.818941 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 619/1244] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:31:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:31:57.824515 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabf5a2d-8046-4857-80f4-62d826d4e08f None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:57.937013 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabf5a2d-8046-4857-80f4-62d826d4e08f None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:57.973974 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93240a92-e0de-4f07-8755-80e5692b0f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:57.974460 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93240a92-e0de-4f07-8755-80e5692b0f9f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:57.974769 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93240a92-e0de-4f07-8755-80e5692b0f9f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:57.975030 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93240a92-e0de-4f07-8755-80e5692b0f9f None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:57.975693 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93240a92-e0de-4f07-8755-80e5692b0f9f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:31:57.975995 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93240a92-e0de-4f07-8755-80e5692b0f9f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:57.977422 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93240a92-e0de-4f07-8755-80e5692b0f9f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:57.989789 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 626/1245] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 15:31:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:31:57.996212 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea46136f-e853-4f73-9855-281f48987d07 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:31:58.108895 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea46136f-e853-4f73-9855-281f48987d07 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:31:58.146057 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81b12e-60fe-4ecd-b407-6d232e393472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:31:58.146751 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81b12e-60fe-4ecd-b407-6d232e393472 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:31:58.147060 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81b12e-60fe-4ecd-b407-6d232e393472 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:31:58.147347 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81b12e-60fe-4ecd-b407-6d232e393472 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:31:58.148195 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce81b12e-60fe-4ecd-b407-6d232e393472 None admin] RBAC: Adding query filter params (role.id=342b64fe0cdb4925b23c6cd461a152ac, scope.project.id=3dc6c5f5fcd34922b0d09cd2aaed6cf1, user.id=de934d0105bd4d01ba6c06a4a0d36704) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:31:58.148528 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce81b12e-60fe-4ecd-b407-6d232e393472 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:31:58.150187 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce81b12e-60fe-4ecd-b407-6d232e393472 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:31:58.156288 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 620/1246] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 15:31:57 2022] GET /identity/v3/role_assignments?user.id=de934d0105bd4d01ba6c06a4a0d36704&scope.project.id=3dc6c5f5fcd34922b0d09cd2aaed6cf1&role.id=342b64fe0cdb4925b23c6cd461a152ac => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:32:00.623882 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82061657-512e-4b30-9c96-d075c61378b6 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:00.624725 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82061657-512e-4b30-9c96-d075c61378b6 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:00.624725 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82061657-512e-4b30-9c96-d075c61378b6 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:00.626057 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 627/1247] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:32:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:32:00.631232 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6e093-5989-4b03-bf9e-9cb5302ba960 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:00.631687 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6e093-5989-4b03-bf9e-9cb5302ba960 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:00.631881 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6e093-5989-4b03-bf9e-9cb5302ba960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:00.638686 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:00.651427 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-dff6e093-5989-4b03-bf9e-9cb5302ba960 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:32:00.652612 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-dff6e093-5989-4b03-bf9e-9cb5302ba960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:32:00.726897 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 621/1248] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:32:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:32:01.035782 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82061657-512e-4b30-9c96-d075c61378b6 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:01.057907 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:01.155129 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82061657-512e-4b30-9c96-d075c61378b6 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:01.196334 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bec7262-399b-4373-bc71-5072052c6f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:01.197244 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec7262-399b-4373-bc71-5072052c6f48 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:01.198227 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec7262-399b-4373-bc71-5072052c6f48 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:01.198565 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec7262-399b-4373-bc71-5072052c6f48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:01.210972 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bec7262-399b-4373-bc71-5072052c6f48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:32:01.216893 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-7bec7262-399b-4373-bc71-5072052c6f48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:32:01.216893 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:32:01.216893 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:32:01.216893 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:32:01.216893 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:32:01.216893 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:32:01.216893 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:32:01.216893 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:32:01.216893 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:32:01.216893 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:32:01.216893 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:32:01.216893 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:32:01.216893 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:32:01.216893 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:32:01.216893 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:32:01.216893 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:32:01.217903 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:32:01.217903 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:32:01.217903 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:32:01.217903 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:32:01.217903 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:32:01.217903 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:32:01.217903 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:32:01.217903 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:32:01.217903 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:32:01.217903 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:32:01.217903 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:32:01.217903 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:32:01.217903 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:32:01.217903 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:32:01.217903 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:32:01.217903 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:32:01.218941 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:32:01.218941 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:32:01.218941 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:32:01.218941 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:32:01.218941 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:32:01.218941 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:32:01.218941 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:32:01.218941 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:32:01.218941 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:32:01.218941 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:32:01.218941 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:32:01.218941 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:32:01.218941 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 628/1249] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 15:32:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:32:01.224368 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff6e093-5989-4b03-bf9e-9cb5302ba960 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:01.331345 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff6e093-5989-4b03-bf9e-9cb5302ba960 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:01.363187 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:01.369810 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab18bab-811b-4b09-b6c1-1432c54170a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:01.370669 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab18bab-811b-4b09-b6c1-1432c54170a2 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:01.370949 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab18bab-811b-4b09-b6c1-1432c54170a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:01.371191 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab18bab-811b-4b09-b6c1-1432c54170a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:01.371851 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab18bab-811b-4b09-b6c1-1432c54170a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:32:01.372170 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab18bab-811b-4b09-b6c1-1432c54170a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:01.373491 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab18bab-811b-4b09-b6c1-1432c54170a2 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:01.382555 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 622/1250] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:32:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:32:01.389304 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bec7262-399b-4373-bc71-5072052c6f48 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:01.473703 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:01.499394 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bec7262-399b-4373-bc71-5072052c6f48 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:01.534657 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45573a5a-b84b-4898-9054-54098367ab3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:01.535311 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45573a5a-b84b-4898-9054-54098367ab3d None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:01.535589 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45573a5a-b84b-4898-9054-54098367ab3d None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:01.535860 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45573a5a-b84b-4898-9054-54098367ab3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:01.547556 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45573a5a-b84b-4898-9054-54098367ab3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:01.548945 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45573a5a-b84b-4898-9054-54098367ab3d None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:01.565951 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-45573a5a-b84b-4898-9054-54098367ab3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:32:01.565951 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:32:01.565951 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:32:01.565951 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:32:01.565951 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:32:01.565951 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:32:01.565951 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:32:01.565951 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:32:01.565951 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:32:01.565951 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:32:01.565951 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:32:01.565951 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:32:01.565951 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:32:01.565951 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:32:01.565951 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:32:01.565951 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:32:01.567079 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:32:01.567079 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:32:01.567079 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:32:01.567079 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:32:01.567079 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:32:01.567079 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:32:01.567079 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:32:01.567079 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:32:01.567079 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:32:01.567079 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:32:01.567079 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:32:01.567079 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:32:01.567079 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:32:01.567079 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:32:01.567079 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:32:01.567079 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:32:01.567079 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:32:01.572920 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:32:01.572920 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:32:01.572920 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:32:01.572920 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 629/1251] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:32:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:32:01.573173 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab18bab-811b-4b09-b6c1-1432c54170a2 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:01.677094 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab18bab-811b-4b09-b6c1-1432c54170a2 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:01.711777 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cc713c-0515-467a-9e6e-8002927be0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:01.712392 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc713c-0515-467a-9e6e-8002927be0ea None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:01.712669 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc713c-0515-467a-9e6e-8002927be0ea None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:01.712937 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc713c-0515-467a-9e6e-8002927be0ea None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:01.713578 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cc713c-0515-467a-9e6e-8002927be0ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:32:01.713906 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cc713c-0515-467a-9e6e-8002927be0ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:01.715289 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cc713c-0515-467a-9e6e-8002927be0ea None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:01.728235 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 623/1252] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:32:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:32:01.734515 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45573a5a-b84b-4898-9054-54098367ab3d None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:01.840321 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45573a5a-b84b-4898-9054-54098367ab3d None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:01.868898 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:01.879914 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25b9be8-0fec-4b2c-8530-f0a332e26572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:01.880598 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25b9be8-0fec-4b2c-8530-f0a332e26572 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:01.880935 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25b9be8-0fec-4b2c-8530-f0a332e26572 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:01.881264 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25b9be8-0fec-4b2c-8530-f0a332e26572 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:01.889488 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25b9be8-0fec-4b2c-8530-f0a332e26572 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:01.890968 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25b9be8-0fec-4b2c-8530-f0a332e26572 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:01.903905 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-e25b9be8-0fec-4b2c-8530-f0a332e26572 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 28 15:32:01.903905 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:32:01.903905 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:32:01.903905 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:32:01.903905 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:32:01.903905 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:32:01.903905 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:32:01.903905 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:32:01.903905 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:32:01.903905 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:32:01.903905 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:32:01.903905 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:32:01.903905 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:32:01.903905 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:32:01.903905 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:32:01.903905 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:32:01.904911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:32:01.904911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:32:01.904911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:32:01.904911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:32:01.904911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:32:01.904911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:32:01.904911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:32:01.904911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:32:01.904911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:32:01.904911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:32:01.904911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:32:01.904911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:32:01.904911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:32:01.904911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:32:01.904911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:32:01.904911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:32:01.904911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:32:01.904911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:32:01.905957 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:32:01.905957 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:32:01.905957 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:32:01.905957 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:32:01.905957 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:32:01.905957 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:32:01.905957 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:32:01.905957 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:32:01.905957 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 28 15:32:01.905957 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:32:01.905957 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 630/1253] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:32:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 28 15:32:01.910938 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cc713c-0515-467a-9e6e-8002927be0ea None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:01.968641 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:02.017372 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cc713c-0515-467a-9e6e-8002927be0ea None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:02.058827 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0428acb-f04a-4b9b-a683-2b5b3b78f4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:02.059434 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0428acb-f04a-4b9b-a683-2b5b3b78f4d8 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:02.059705 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0428acb-f04a-4b9b-a683-2b5b3b78f4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:02.059968 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0428acb-f04a-4b9b-a683-2b5b3b78f4d8 None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:02.060695 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0428acb-f04a-4b9b-a683-2b5b3b78f4d8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:32:02.060993 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0428acb-f04a-4b9b-a683-2b5b3b78f4d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:02.062426 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0428acb-f04a-4b9b-a683-2b5b3b78f4d8 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:02.079558 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 624/1254] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Sep 28 15:32:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:32:02.085284 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25b9be8-0fec-4b2c-8530-f0a332e26572 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:02.188395 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25b9be8-0fec-4b2c-8530-f0a332e26572 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:02.219801 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c12ed8-1ffb-4236-a7dd-3458b498769c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:02.220385 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c12ed8-1ffb-4236-a7dd-3458b498769c None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:02.220659 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c12ed8-1ffb-4236-a7dd-3458b498769c None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:02.220923 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c12ed8-1ffb-4236-a7dd-3458b498769c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:02.273168 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:02.285381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c12ed8-1ffb-4236-a7dd-3458b498769c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:02.286841 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c12ed8-1ffb-4236-a7dd-3458b498769c None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:02.379819 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 631/1255] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:32:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 15:32:02.385567 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0428acb-f04a-4b9b-a683-2b5b3b78f4d8 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:02.482943 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0428acb-f04a-4b9b-a683-2b5b3b78f4d8 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:02.495731 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:02.516084 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17b801e-fc46-4f50-bebe-955cb19a81a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:02.516696 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b801e-fc46-4f50-bebe-955cb19a81a8 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:02.516968 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b801e-fc46-4f50-bebe-955cb19a81a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:02.517232 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b801e-fc46-4f50-bebe-955cb19a81a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:02.526746 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17b801e-fc46-4f50-bebe-955cb19a81a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:02.528159 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17b801e-fc46-4f50-bebe-955cb19a81a8 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:02.542587 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-d17b801e-fc46-4f50-bebe-955cb19a81a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:32:02.542587 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:32:02.542587 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:32:02.542587 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:32:02.542587 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:32:02.542587 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:32:02.542587 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:32:02.542587 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:32:02.542587 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:32:02.542587 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:32:02.542587 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:32:02.542587 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:32:02.542587 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:32:02.542587 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:32:02.542587 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:32:02.542587 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:32:02.543676 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:32:02.543676 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:32:02.543676 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:32:02.543676 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:32:02.543676 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:32:02.543676 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:32:02.543676 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:32:02.543676 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:32:02.543676 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:32:02.543676 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:32:02.543676 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:32:02.543676 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:32:02.543676 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:32:02.543676 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:32:02.543676 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:32:02.543676 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:32:02.543676 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:32:02.544563 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:32:02.544563 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:32:02.544563 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:32:02.544563 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 625/1256] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:32:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:32:02.549369 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c12ed8-1ffb-4236-a7dd-3458b498769c None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:02.631481 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:02.643250 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c12ed8-1ffb-4236-a7dd-3458b498769c None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:02.675714 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04186f7-dcd3-4200-9a29-7e45420ba938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:02.676293 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04186f7-dcd3-4200-9a29-7e45420ba938 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:02.676568 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04186f7-dcd3-4200-9a29-7e45420ba938 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:02.676837 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04186f7-dcd3-4200-9a29-7e45420ba938 None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:02.677453 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04186f7-dcd3-4200-9a29-7e45420ba938 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:32:02.677749 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04186f7-dcd3-4200-9a29-7e45420ba938 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:02.679126 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04186f7-dcd3-4200-9a29-7e45420ba938 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:02.689711 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 632/1257] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:32:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:32:02.695170 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17b801e-fc46-4f50-bebe-955cb19a81a8 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:02.790702 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17b801e-fc46-4f50-bebe-955cb19a81a8 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:02.823342 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b8f5ef-3838-4f43-a8bd-1c230d9f9346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:02.823943 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8f5ef-3838-4f43-a8bd-1c230d9f9346 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:02.824215 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8f5ef-3838-4f43-a8bd-1c230d9f9346 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:02.824480 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8f5ef-3838-4f43-a8bd-1c230d9f9346 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:02.830717 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b8f5ef-3838-4f43-a8bd-1c230d9f9346 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:02.832137 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b8f5ef-3838-4f43-a8bd-1c230d9f9346 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:02.844108 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-87b8f5ef-3838-4f43-a8bd-1c230d9f9346 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:32:02.844108 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:32:02.844108 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:32:02.844108 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:32:02.844108 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:32:02.844108 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:32:02.844108 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:32:02.844108 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:32:02.844108 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:32:02.844108 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:32:02.844108 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:32:02.844108 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:32:02.844108 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:32:02.844108 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:32:02.844108 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:32:02.845177 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:32:02.845177 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:32:02.845177 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:32:02.845177 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:32:02.845177 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:32:02.845177 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:32:02.845177 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:32:02.845177 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:32:02.845177 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:32:02.845177 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:32:02.845177 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:32:02.845177 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:32:02.845177 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:32:02.845177 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:32:02.845177 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:32:02.845177 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:32:02.845177 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:32:02.846144 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:32:02.846144 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:32:02.846144 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:32:02.846144 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:32:02.846144 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:32:02.846144 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:32:02.846144 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:32:02.846144 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:32:02.846144 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 626/1258] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:32:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:32:02.850899 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04186f7-dcd3-4200-9a29-7e45420ba938 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:02.947252 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04186f7-dcd3-4200-9a29-7e45420ba938 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:02.980455 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8136d0e-b61f-47cf-a666-73f770666d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:02.981048 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8136d0e-b61f-47cf-a666-73f770666d8f None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:02.981328 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8136d0e-b61f-47cf-a666-73f770666d8f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:02.981597 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8136d0e-b61f-47cf-a666-73f770666d8f None admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:02.982303 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8136d0e-b61f-47cf-a666-73f770666d8f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:32:02.982607 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8136d0e-b61f-47cf-a666-73f770666d8f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:02.984026 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8136d0e-b61f-47cf-a666-73f770666d8f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:02.997022 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 633/1259] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 15:32:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:32:03.003585 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b8f5ef-3838-4f43-a8bd-1c230d9f9346 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:03.033687 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:03.106426 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b8f5ef-3838-4f43-a8bd-1c230d9f9346 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:03.139157 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cc3fe0-61fd-4d7f-b1f9-6e3ed86a039f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:03.139758 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc3fe0-61fd-4d7f-b1f9-6e3ed86a039f None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:03.140037 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc3fe0-61fd-4d7f-b1f9-6e3ed86a039f None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:03.140309 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc3fe0-61fd-4d7f-b1f9-6e3ed86a039f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:03.141025 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cc3fe0-61fd-4d7f-b1f9-6e3ed86a039f None admin] RBAC: Adding query filter params (role.id=62d96ac9513c452894fc67b442cfc59f, scope.project.id=3dc6c5f5fcd34922b0d09cd2aaed6cf1, user.id=de934d0105bd4d01ba6c06a4a0d36704) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:32:03.141334 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cc3fe0-61fd-4d7f-b1f9-6e3ed86a039f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:03.142791 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cc3fe0-61fd-4d7f-b1f9-6e3ed86a039f None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:03.148181 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 627/1260] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 15:32:03 2022] GET /identity/v3/role_assignments?user.id=de934d0105bd4d01ba6c06a4a0d36704&scope.project.id=3dc6c5f5fcd34922b0d09cd2aaed6cf1&role.id=62d96ac9513c452894fc67b442cfc59f => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:32:05.279185 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab0db4-dc08-4f45-847f-04a7d74742ec None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:05.279727 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab0db4-dc08-4f45-847f-04a7d74742ec None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:05.279727 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab0db4-dc08-4f45-847f-04a7d74742ec None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:05.281058 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 634/1261] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:32:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:32:05.286124 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90edb11a-dd2c-48e2-af8a-f44e9c9a4281 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:05.286511 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90edb11a-dd2c-48e2-af8a-f44e9c9a4281 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:05.286672 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90edb11a-dd2c-48e2-af8a-f44e9c9a4281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:05.305282 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-90edb11a-dd2c-48e2-af8a-f44e9c9a4281 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:32:05.306522 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-90edb11a-dd2c-48e2-af8a-f44e9c9a4281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:32:05.372574 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 628/1262] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:32:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:32:05.645107 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbab0db4-dc08-4f45-847f-04a7d74742ec None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:05.656319 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:05.744092 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbab0db4-dc08-4f45-847f-04a7d74742ec None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:05.776652 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a36640d-51ae-4b99-8ee7-c24e12078644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:05.777267 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36640d-51ae-4b99-8ee7-c24e12078644 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:05.777548 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36640d-51ae-4b99-8ee7-c24e12078644 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:05.777821 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36640d-51ae-4b99-8ee7-c24e12078644 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:05.787503 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a36640d-51ae-4b99-8ee7-c24e12078644 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:32:05.792864 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-2a36640d-51ae-4b99-8ee7-c24e12078644 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:32:05.792864 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:32:05.792864 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:32:05.792864 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:32:05.792864 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:32:05.792864 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:32:05.792864 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:32:05.792864 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:32:05.792864 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:32:05.792864 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:32:05.792864 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:32:05.792864 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:32:05.792864 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:32:05.792864 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:32:05.792864 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:32:05.792864 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:32:05.793973 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:32:05.793973 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:32:05.793973 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:32:05.793973 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:32:05.793973 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:32:05.793973 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:32:05.793973 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:32:05.793973 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:32:05.793973 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:32:05.793973 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:32:05.793973 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:32:05.793973 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:32:05.793973 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:32:05.793973 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:32:05.793973 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:32:05.793973 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:32:05.795163 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:32:05.795163 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:32:05.795163 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:32:05.795163 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:32:05.795163 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:32:05.795163 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:32:05.795163 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:32:05.795163 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:32:05.795163 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:32:05.795163 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:32:05.795163 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:32:05.795163 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:32:05.795163 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 635/1263] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 15:32:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:32:05.800181 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90edb11a-dd2c-48e2-af8a-f44e9c9a4281 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:05.817719 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:05.894708 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90edb11a-dd2c-48e2-af8a-f44e9c9a4281 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:05.927324 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a5ef0-2289-440b-88ff-8d0a825eeeff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:05.927855 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a5ef0-2289-440b-88ff-8d0a825eeeff None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:05.928129 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a5ef0-2289-440b-88ff-8d0a825eeeff None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:05.928398 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a5ef0-2289-440b-88ff-8d0a825eeeff None admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:05.929040 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03a5ef0-2289-440b-88ff-8d0a825eeeff None admin] RBAC: Adding query filter params (name=admin) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:32:05.929339 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03a5ef0-2289-440b-88ff-8d0a825eeeff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:05.930702 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03a5ef0-2289-440b-88ff-8d0a825eeeff None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:05.938366 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 629/1264] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:32:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:32:05.943994 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a36640d-51ae-4b99-8ee7-c24e12078644 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:06.014678 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:06.052253 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a36640d-51ae-4b99-8ee7-c24e12078644 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:06.087755 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db2eb2-f774-4cce-b2fc-65f198c0eea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:06.088893 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db2eb2-f774-4cce-b2fc-65f198c0eea5 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:06.089171 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db2eb2-f774-4cce-b2fc-65f198c0eea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:06.089438 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db2eb2-f774-4cce-b2fc-65f198c0eea5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:06.099068 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93db2eb2-f774-4cce-b2fc-65f198c0eea5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:06.100512 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93db2eb2-f774-4cce-b2fc-65f198c0eea5 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:06.115263 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-93db2eb2-f774-4cce-b2fc-65f198c0eea5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:32:06.115263 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:32:06.115263 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:32:06.115263 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:32:06.115263 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:32:06.115263 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:32:06.115263 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:32:06.115263 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:32:06.115263 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:32:06.115263 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:32:06.115263 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:32:06.115263 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:32:06.115263 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:32:06.115263 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:32:06.115263 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:32:06.115263 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:32:06.116390 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:32:06.116390 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:32:06.116390 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:32:06.116390 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:32:06.116390 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:32:06.116390 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:32:06.116390 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:32:06.116390 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:32:06.116390 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:32:06.116390 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:32:06.116390 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:32:06.116390 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:32:06.116390 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:32:06.116390 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:32:06.116390 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:32:06.116390 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:32:06.116390 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:32:06.117489 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:32:06.117489 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:32:06.117489 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:32:06.117489 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 636/1265] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:32:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:32:06.122277 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a5ef0-2289-440b-88ff-8d0a825eeeff None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:06.200295 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:06.219537 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a5ef0-2289-440b-88ff-8d0a825eeeff None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:06.251463 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2f02f3-d75b-45d6-8da0-d38cafa802af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:06.252051 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f02f3-d75b-45d6-8da0-d38cafa802af None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:06.252325 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f02f3-d75b-45d6-8da0-d38cafa802af None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:06.252590 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f02f3-d75b-45d6-8da0-d38cafa802af None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:06.253213 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2f02f3-d75b-45d6-8da0-d38cafa802af None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:32:06.253511 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2f02f3-d75b-45d6-8da0-d38cafa802af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:06.254889 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2f02f3-d75b-45d6-8da0-d38cafa802af None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:06.266537 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 630/1266] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:32:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:32:06.272249 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db2eb2-f774-4cce-b2fc-65f198c0eea5 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:06.366710 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db2eb2-f774-4cce-b2fc-65f198c0eea5 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:06.386420 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:06.400003 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9917915b-01a1-4de1-bcbb-0e75e732a06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:06.400618 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917915b-01a1-4de1-bcbb-0e75e732a06e None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:06.400895 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917915b-01a1-4de1-bcbb-0e75e732a06e None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:06.401166 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917915b-01a1-4de1-bcbb-0e75e732a06e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:06.408253 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9917915b-01a1-4de1-bcbb-0e75e732a06e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:06.409683 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9917915b-01a1-4de1-bcbb-0e75e732a06e None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:06.420476 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-9917915b-01a1-4de1-bcbb-0e75e732a06e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 28 15:32:06.420476 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:32:06.420476 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:32:06.420476 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:32:06.420476 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:32:06.420476 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:32:06.420476 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:32:06.420476 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:32:06.420476 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:32:06.420476 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:32:06.420476 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:32:06.420476 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:32:06.420476 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:32:06.420476 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:32:06.420476 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:32:06.420476 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:32:06.421592 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:32:06.421592 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:32:06.421592 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:32:06.421592 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:32:06.421592 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:32:06.421592 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:32:06.421592 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:32:06.421592 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:32:06.421592 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:32:06.421592 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:32:06.421592 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:32:06.421592 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:32:06.421592 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:32:06.421592 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:32:06.421592 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:32:06.421592 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:32:06.421592 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:32:06.421592 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:32:06.422786 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:32:06.422786 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:32:06.422786 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:32:06.422786 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:32:06.422786 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:32:06.422786 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:32:06.422786 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:32:06.422786 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:32:06.422786 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 28 15:32:06.422786 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:32:06.422786 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 637/1267] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:32:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:32:06.427435 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2f02f3-d75b-45d6-8da0-d38cafa802af None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:06.523151 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2f02f3-d75b-45d6-8da0-d38cafa802af None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:06.554798 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89a38f0-3f27-4596-aad6-e993797198a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:06.555384 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a38f0-3f27-4596-aad6-e993797198a1 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:06.555660 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a38f0-3f27-4596-aad6-e993797198a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:06.555925 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a38f0-3f27-4596-aad6-e993797198a1 None admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:06.556626 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89a38f0-3f27-4596-aad6-e993797198a1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:32:06.556923 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89a38f0-3f27-4596-aad6-e993797198a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:06.558879 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89a38f0-3f27-4596-aad6-e993797198a1 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:06.567092 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:06.577422 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 631/1268] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Sep 28 15:32:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:32:06.583186 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9917915b-01a1-4de1-bcbb-0e75e732a06e None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:06.678072 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9917915b-01a1-4de1-bcbb-0e75e732a06e None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:06.710323 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee67d05-d6c7-42d7-86e2-da845451258a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:06.710925 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee67d05-d6c7-42d7-86e2-da845451258a None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:06.711199 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee67d05-d6c7-42d7-86e2-da845451258a None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:06.711466 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee67d05-d6c7-42d7-86e2-da845451258a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:06.721303 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee67d05-d6c7-42d7-86e2-da845451258a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:06.722740 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee67d05-d6c7-42d7-86e2-da845451258a None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:06.736516 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-0ee67d05-d6c7-42d7-86e2-da845451258a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:32:06.736516 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:32:06.736516 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:32:06.736516 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:32:06.736516 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:32:06.736516 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:32:06.736516 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:32:06.736516 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:32:06.736516 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:32:06.736516 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:32:06.736516 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:32:06.736516 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:32:06.736516 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:32:06.736516 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:32:06.736516 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:32:06.736516 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:32:06.737629 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:32:06.737629 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:32:06.737629 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:32:06.737629 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:32:06.737629 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:32:06.737629 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:32:06.737629 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:32:06.737629 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:32:06.737629 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:32:06.737629 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:32:06.737629 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:32:06.737629 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:32:06.737629 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:32:06.737629 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:32:06.737629 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:32:06.737629 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:32:06.737629 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:32:06.738750 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:32:06.738750 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:32:06.738750 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:32:06.738750 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 638/1269] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:32:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:32:06.743237 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89a38f0-3f27-4596-aad6-e993797198a1 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:06.841751 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89a38f0-3f27-4596-aad6-e993797198a1 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:06.875663 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af634e5f-32af-4933-bcb8-aabf83f7d044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:06.876256 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af634e5f-32af-4933-bcb8-aabf83f7d044 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:06.876530 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af634e5f-32af-4933-bcb8-aabf83f7d044 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:06.876797 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af634e5f-32af-4933-bcb8-aabf83f7d044 None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:06.877420 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af634e5f-32af-4933-bcb8-aabf83f7d044 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:32:06.877718 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af634e5f-32af-4933-bcb8-aabf83f7d044 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:06.879106 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af634e5f-32af-4933-bcb8-aabf83f7d044 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:06.889824 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 632/1270] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:32:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:32:06.895474 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee67d05-d6c7-42d7-86e2-da845451258a None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:06.925678 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:06.992612 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee67d05-d6c7-42d7-86e2-da845451258a None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:07.028665 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db336bee-86e0-465e-9173-843c580e2ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:07.029282 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db336bee-86e0-465e-9173-843c580e2ba4 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:07.029558 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db336bee-86e0-465e-9173-843c580e2ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:07.029826 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db336bee-86e0-465e-9173-843c580e2ba4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:07.036200 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db336bee-86e0-465e-9173-843c580e2ba4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:07.037644 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db336bee-86e0-465e-9173-843c580e2ba4 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:07.048490 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-db336bee-86e0-465e-9173-843c580e2ba4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:32:07.048490 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:32:07.048490 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:32:07.048490 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:32:07.048490 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:32:07.048490 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:32:07.048490 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:32:07.048490 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:32:07.048490 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:32:07.048490 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:32:07.048490 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:32:07.048490 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:32:07.048490 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:32:07.048490 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:32:07.048490 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:32:07.049388 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:32:07.049388 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:32:07.049388 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:32:07.049388 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:32:07.049388 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:32:07.049388 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:32:07.049388 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:32:07.049388 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:32:07.049388 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:32:07.049388 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:32:07.049388 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:32:07.049388 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:32:07.049388 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:32:07.049388 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:32:07.049388 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:32:07.049388 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:32:07.049388 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:32:07.050502 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:32:07.050502 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:32:07.050502 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:32:07.050502 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:32:07.050502 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:32:07.050502 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:32:07.050502 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:32:07.050502 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:32:07.050502 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 639/1271] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:32:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:32:07.055418 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af634e5f-32af-4933-bcb8-aabf83f7d044 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:07.111134 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:07.156596 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af634e5f-32af-4933-bcb8-aabf83f7d044 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:07.200868 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de409f49-7c78-4fa8-b36f-df9b82bb932d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:07.201469 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de409f49-7c78-4fa8-b36f-df9b82bb932d None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:07.201743 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de409f49-7c78-4fa8-b36f-df9b82bb932d None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:07.202042 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de409f49-7c78-4fa8-b36f-df9b82bb932d None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:07.202720 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de409f49-7c78-4fa8-b36f-df9b82bb932d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:32:07.203022 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de409f49-7c78-4fa8-b36f-df9b82bb932d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:07.204434 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de409f49-7c78-4fa8-b36f-df9b82bb932d None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:07.216217 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 633/1272] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 15:32:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:32:07.222227 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db336bee-86e0-465e-9173-843c580e2ba4 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:07.313405 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:07.322022 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db336bee-86e0-465e-9173-843c580e2ba4 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:07.354373 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6253ec4-e7a6-4503-877f-73901841ce2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:07.355013 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6253ec4-e7a6-4503-877f-73901841ce2f None admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:07.355292 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6253ec4-e7a6-4503-877f-73901841ce2f None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:07.355564 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6253ec4-e7a6-4503-877f-73901841ce2f None admin] PATH_INFO: `/v3/projects/3dc6c5f5fcd34922b0d09cd2aaed6cf1/users/de934d0105bd4d01ba6c06a4a0d36704/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:07.357412 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6253ec4-e7a6-4503-877f-73901841ce2f None admin] RBAC: Authorizing `identity:create_grant(project_id=3dc6c5f5fcd34922b0d09cd2aaed6cf1, user_id=de934d0105bd4d01ba6c06a4a0d36704, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:07.358862 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6253ec4-e7a6-4503-877f-73901841ce2f None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:07.371394 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 640/1273] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Sep 28 15:32:07 2022] PUT /identity/v3/projects/3dc6c5f5fcd34922b0d09cd2aaed6cf1/users/de934d0105bd4d01ba6c06a4a0d36704/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:32:09.507515 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405fae5f-9ac2-49fe-8acc-623eaaf66d37 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:09.508067 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405fae5f-9ac2-49fe-8acc-623eaaf66d37 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:09.508067 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405fae5f-9ac2-49fe-8acc-623eaaf66d37 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:09.509397 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 634/1274] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:32:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:32:09.516304 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c42dc-77e9-4d28-91eb-8608a318196d None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:09.516697 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c42dc-77e9-4d28-91eb-8608a318196d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:09.516862 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c42dc-77e9-4d28-91eb-8608a318196d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:09.535152 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-430c42dc-77e9-4d28-91eb-8608a318196d None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:32:09.536332 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-430c42dc-77e9-4d28-91eb-8608a318196d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:32:09.600263 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 641/1275] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:32:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:32:09.866639 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405fae5f-9ac2-49fe-8acc-623eaaf66d37 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:09.962514 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405fae5f-9ac2-49fe-8acc-623eaaf66d37 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:09.979080 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:09.997029 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e1bb4b-c223-4d36-9133-51fb2ef0ace1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:09.997678 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e1bb4b-c223-4d36-9133-51fb2ef0ace1 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:09.997996 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e1bb4b-c223-4d36-9133-51fb2ef0ace1 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:09.998275 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e1bb4b-c223-4d36-9133-51fb2ef0ace1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:10.009023 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e1bb4b-c223-4d36-9133-51fb2ef0ace1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 28 15:32:10.014651 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-91e1bb4b-c223-4d36-9133-51fb2ef0ace1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:32:10.014651 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:32:10.014651 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:32:10.014651 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:32:10.014651 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:32:10.014651 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:32:10.014651 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:32:10.014651 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:32:10.014651 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:32:10.014651 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:32:10.014651 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:32:10.014651 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:32:10.014651 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 28 15:32:10.014651 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 28 15:32:10.014651 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 28 15:32:10.014651 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 28 15:32:10.016003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 28 15:32:10.016003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 28 15:32:10.016003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 28 15:32:10.016003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 28 15:32:10.016003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:32:10.016003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:32:10.016003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:32:10.016003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:32:10.016003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:32:10.016003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:32:10.016003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:32:10.016003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:32:10.016003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:32:10.016003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:32:10.016003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:32:10.016003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:32:10.016887 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:32:10.016887 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:32:10.016887 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:32:10.016887 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:32:10.016887 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 28 15:32:10.016887 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 28 15:32:10.016887 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 28 15:32:10.016887 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 28 15:32:10.016887 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 28 15:32:10.016887 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 28 15:32:10.016887 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 28 15:32:10.016887 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:32:10.016887 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 635/1276] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Sep 28 15:32:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:32:10.021919 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430c42dc-77e9-4d28-91eb-8608a318196d None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:10.103848 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:10.122255 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430c42dc-77e9-4d28-91eb-8608a318196d None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:10.154385 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b5b5ef-d174-43d2-a7be-552b43f364d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:10.154973 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b5b5ef-d174-43d2-a7be-552b43f364d0 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:10.155250 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b5b5ef-d174-43d2-a7be-552b43f364d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:10.155515 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b5b5ef-d174-43d2-a7be-552b43f364d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:10.156158 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b5b5ef-d174-43d2-a7be-552b43f364d0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:32:10.156454 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b5b5ef-d174-43d2-a7be-552b43f364d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:10.157806 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b5b5ef-d174-43d2-a7be-552b43f364d0 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:10.165421 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 642/1277] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Sep 28 15:32:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:32:10.172181 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e1bb4b-c223-4d36-9133-51fb2ef0ace1 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:10.269110 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e1bb4b-c223-4d36-9133-51fb2ef0ace1 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:10.302262 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf4897b-ebf7-46ff-b2db-612364fa3460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:10.302884 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf4897b-ebf7-46ff-b2db-612364fa3460 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:10.303162 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf4897b-ebf7-46ff-b2db-612364fa3460 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:10.303433 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf4897b-ebf7-46ff-b2db-612364fa3460 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:10.313627 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf4897b-ebf7-46ff-b2db-612364fa3460 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:10.315120 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf4897b-ebf7-46ff-b2db-612364fa3460 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:10.329836 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-bbf4897b-ebf7-46ff-b2db-612364fa3460 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:32:10.329836 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:32:10.329836 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:32:10.329836 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:32:10.329836 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:32:10.329836 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:32:10.329836 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:32:10.329836 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:32:10.329836 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:32:10.329836 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:32:10.329836 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:32:10.329836 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:32:10.329836 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:32:10.329836 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:32:10.329836 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:32:10.329836 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:32:10.330987 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:32:10.330987 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:32:10.330987 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:32:10.330987 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:32:10.330987 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:32:10.330987 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:32:10.330987 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:32:10.330987 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:32:10.330987 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:32:10.330987 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:32:10.330987 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:32:10.330987 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:32:10.330987 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:32:10.330987 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:32:10.330987 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:32:10.330987 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:32:10.330987 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:32:10.331883 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:32:10.331883 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:32:10.331883 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:32:10.331883 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 636/1278] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:32:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:32:10.336684 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b5b5ef-d174-43d2-a7be-552b43f364d0 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:10.431953 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b5b5ef-d174-43d2-a7be-552b43f364d0 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:10.464619 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a5c2d5-a946-42be-a9b3-948f0dbc91ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:10.465204 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a5c2d5-a946-42be-a9b3-948f0dbc91ac None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:10.465484 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a5c2d5-a946-42be-a9b3-948f0dbc91ac None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:10.465758 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a5c2d5-a946-42be-a9b3-948f0dbc91ac None admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:10.466423 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a5c2d5-a946-42be-a9b3-948f0dbc91ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:32:10.466726 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a5c2d5-a946-42be-a9b3-948f0dbc91ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:10.468090 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a5c2d5-a946-42be-a9b3-948f0dbc91ac None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:10.479756 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 643/1279] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:32:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:32:10.485256 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf4897b-ebf7-46ff-b2db-612364fa3460 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:10.516361 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:10.582663 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf4897b-ebf7-46ff-b2db-612364fa3460 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:10.614000 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1267fba8-d978-4d68-9ff6-36240d59bcb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:10.614599 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1267fba8-d978-4d68-9ff6-36240d59bcb5 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:10.614869 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1267fba8-d978-4d68-9ff6-36240d59bcb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:10.615134 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1267fba8-d978-4d68-9ff6-36240d59bcb5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:10.622670 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1267fba8-d978-4d68-9ff6-36240d59bcb5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:10.624345 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1267fba8-d978-4d68-9ff6-36240d59bcb5 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:10.634950 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-1267fba8-d978-4d68-9ff6-36240d59bcb5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 28 15:32:10.634950 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:32:10.634950 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:32:10.634950 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:32:10.634950 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:32:10.634950 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:32:10.634950 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:32:10.634950 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:32:10.634950 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:32:10.634950 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:32:10.634950 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:32:10.634950 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:32:10.634950 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 28 15:32:10.634950 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 28 15:32:10.634950 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 28 15:32:10.634950 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 28 15:32:10.636071 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:32:10.636071 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:32:10.636071 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 28 15:32:10.636071 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:32:10.636071 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 28 15:32:10.636071 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 28 15:32:10.636071 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:32:10.636071 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:32:10.636071 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:32:10.636071 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:32:10.636071 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:32:10.636071 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:32:10.636071 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:32:10.636071 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:32:10.636071 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:32:10.636071 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:32:10.636071 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:32:10.636071 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:32:10.637381 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:32:10.637381 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:32:10.637381 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 28 15:32:10.637381 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 28 15:32:10.637381 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 28 15:32:10.637381 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 28 15:32:10.637381 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 28 15:32:10.637381 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 28 15:32:10.637381 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 28 15:32:10.637381 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:32:10.637381 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 637/1280] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:32:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:32:10.641693 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a5c2d5-a946-42be-a9b3-948f0dbc91ac None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:10.645380 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:10.735841 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a5c2d5-a946-42be-a9b3-948f0dbc91ac None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:10.767728 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2492e529-84bc-4e4d-9c1f-5c626180a5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:10.768307 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492e529-84bc-4e4d-9c1f-5c626180a5d9 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:10.768583 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492e529-84bc-4e4d-9c1f-5c626180a5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:10.768852 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2492e529-84bc-4e4d-9c1f-5c626180a5d9 None admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:10.769550 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2492e529-84bc-4e4d-9c1f-5c626180a5d9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:32:10.769871 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2492e529-84bc-4e4d-9c1f-5c626180a5d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:10.771276 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2492e529-84bc-4e4d-9c1f-5c626180a5d9 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:10.787287 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 644/1281] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Sep 28 15:32:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:32:10.793429 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1267fba8-d978-4d68-9ff6-36240d59bcb5 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:10.881227 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:10.890769 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1267fba8-d978-4d68-9ff6-36240d59bcb5 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:10.923484 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f915905e-7df3-4eac-9486-51801419ac72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:10.924075 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f915905e-7df3-4eac-9486-51801419ac72 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:10.924352 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f915905e-7df3-4eac-9486-51801419ac72 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:10.924623 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f915905e-7df3-4eac-9486-51801419ac72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:10.989292 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f915905e-7df3-4eac-9486-51801419ac72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:10.990845 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f915905e-7df3-4eac-9486-51801419ac72 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:11.076009 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:11.085833 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 638/1282] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Sep 28 15:32:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 28 15:32:11.091664 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2492e529-84bc-4e4d-9c1f-5c626180a5d9 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:11.149736 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:11.186690 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2492e529-84bc-4e4d-9c1f-5c626180a5d9 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:11.218708 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c51309-63b8-42f5-ac88-a52bfbfe71d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:11.219314 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c51309-63b8-42f5-ac88-a52bfbfe71d9 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:11.219592 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c51309-63b8-42f5-ac88-a52bfbfe71d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:11.219859 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c51309-63b8-42f5-ac88-a52bfbfe71d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:11.229821 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c51309-63b8-42f5-ac88-a52bfbfe71d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:11.231280 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c51309-63b8-42f5-ac88-a52bfbfe71d9 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:11.245802 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-41c51309-63b8-42f5-ac88-a52bfbfe71d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:32:11.245802 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:32:11.245802 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:32:11.245802 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:32:11.245802 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:32:11.245802 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:32:11.245802 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:32:11.245802 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:32:11.245802 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:32:11.245802 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:32:11.245802 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:32:11.245802 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:32:11.245802 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 28 15:32:11.245802 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 28 15:32:11.245802 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 28 15:32:11.245802 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 28 15:32:11.246948 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:32:11.246948 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:32:11.246948 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:32:11.246948 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:32:11.246948 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:32:11.246948 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:32:11.246948 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:32:11.246948 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:32:11.246948 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:32:11.246948 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:32:11.246948 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:32:11.246948 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:32:11.246948 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:32:11.246948 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:32:11.246948 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:32:11.246948 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:32:11.246948 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 28 15:32:11.248033 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 28 15:32:11.248033 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 28 15:32:11.248033 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:32:11.248033 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 645/1283] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:32:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:32:11.252860 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f915905e-7df3-4eac-9486-51801419ac72 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:11.348287 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f915905e-7df3-4eac-9486-51801419ac72 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:11.381731 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad8180d-fb28-4516-a712-1a7def1324db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:11.382346 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8180d-fb28-4516-a712-1a7def1324db None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:11.382622 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8180d-fb28-4516-a712-1a7def1324db None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:11.382890 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8180d-fb28-4516-a712-1a7def1324db None admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:11.383515 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad8180d-fb28-4516-a712-1a7def1324db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:32:11.383812 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad8180d-fb28-4516-a712-1a7def1324db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:11.385171 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad8180d-fb28-4516-a712-1a7def1324db None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:11.395832 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 639/1284] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Sep 28 15:32:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:32:11.401609 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c51309-63b8-42f5-ac88-a52bfbfe71d9 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:11.497279 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c51309-63b8-42f5-ac88-a52bfbfe71d9 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:11.510250 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:11.529587 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa330c-bada-4b5d-87ee-d325164bed06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:11.530198 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa330c-bada-4b5d-87ee-d325164bed06 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:11.530474 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa330c-bada-4b5d-87ee-d325164bed06 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:11.530741 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa330c-bada-4b5d-87ee-d325164bed06 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:11.536986 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafa330c-bada-4b5d-87ee-d325164bed06 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:11.538436 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafa330c-bada-4b5d-87ee-d325164bed06 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:11.549643 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application [None req-fafa330c-bada-4b5d-87ee-d325164bed06 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:32:11.549643 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:32:11.549643 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:32:11.549643 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:32:11.549643 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:32:11.549643 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:32:11.549643 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:32:11.549643 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:32:11.549643 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:32:11.549643 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:32:11.549643 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:32:11.549643 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:32:11.549643 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 28 15:32:11.549643 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 28 15:32:11.549643 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 28 15:32:11.550724 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 28 15:32:11.550724 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:32:11.550724 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:32:11.550724 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:32:11.550724 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:32:11.550724 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:32:11.550724 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:32:11.550724 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:32:11.550724 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application with Lock( Sep 28 15:32:11.550724 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:32:11.550724 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:32:11.550724 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:32:11.550724 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:32:11.550724 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:32:11.550724 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:32:11.550724 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:32:11.550724 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:32:11.551906 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 28 15:32:11.551906 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 28 15:32:11.551906 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 28 15:32:11.551906 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 28 15:32:11.551906 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 28 15:32:11.551906 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 28 15:32:11.551906 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 28 15:32:11.551906 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: ERROR keystone.server.flask.application Sep 28 15:32:11.551906 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 646/1285] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Sep 28 15:32:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:32:11.556499 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad8180d-fb28-4516-a712-1a7def1324db None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:11.605789 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:11.653601 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad8180d-fb28-4516-a712-1a7def1324db None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:11.685510 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29560966-c04d-4afc-94f0-31d6d51a7d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:11.686118 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29560966-c04d-4afc-94f0-31d6d51a7d67 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:11.686393 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29560966-c04d-4afc-94f0-31d6d51a7d67 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:11.686661 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29560966-c04d-4afc-94f0-31d6d51a7d67 None admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:11.687333 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29560966-c04d-4afc-94f0-31d6d51a7d67 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:32:11.687629 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29560966-c04d-4afc-94f0-31d6d51a7d67 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:11.689038 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29560966-c04d-4afc-94f0-31d6d51a7d67 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:11.700824 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 640/1286] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Sep 28 15:32:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:32:11.706788 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa330c-bada-4b5d-87ee-d325164bed06 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:11.803019 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa330c-bada-4b5d-87ee-d325164bed06 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:11.834148 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c28708-a554-44bc-a227-fcedaee89ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:11.834725 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c28708-a554-44bc-a227-fcedaee89ff3 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:11.834998 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c28708-a554-44bc-a227-fcedaee89ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:11.835263 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c28708-a554-44bc-a227-fcedaee89ff3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:11.835960 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c28708-a554-44bc-a227-fcedaee89ff3 None admin] RBAC: Adding query filter params (role.id=62d96ac9513c452894fc67b442cfc59f, scope.project.id=3dc6c5f5fcd34922b0d09cd2aaed6cf1, user.id=de934d0105bd4d01ba6c06a4a0d36704) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:32:11.836257 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c28708-a554-44bc-a227-fcedaee89ff3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:11.837658 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c28708-a554-44bc-a227-fcedaee89ff3 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:11.843626 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 647/1287] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Sep 28 15:32:11 2022] GET /identity/v3/role_assignments?user.id=de934d0105bd4d01ba6c06a4a0d36704&scope.project.id=3dc6c5f5fcd34922b0d09cd2aaed6cf1&role.id=62d96ac9513c452894fc67b442cfc59f => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:32:14.013642 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c536335c-0c7c-4d1c-8e34-b787de04b37c None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:14.014238 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c536335c-0c7c-4d1c-8e34-b787de04b37c None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:14.014238 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c536335c-0c7c-4d1c-8e34-b787de04b37c None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:14.015618 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 641/1288] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:32:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:32:14.022018 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec948d4-edea-4171-a12f-b9d9941090dd None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:14.022463 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec948d4-edea-4171-a12f-b9d9941090dd None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:14.022746 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec948d4-edea-4171-a12f-b9d9941090dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:14.046013 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-fec948d4-edea-4171-a12f-b9d9941090dd None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:32:14.047485 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-fec948d4-edea-4171-a12f-b9d9941090dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:32:14.080296 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:14.117473 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 648/1289] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:32:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:32:14.417496 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c536335c-0c7c-4d1c-8e34-b787de04b37c None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:14.515453 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c536335c-0c7c-4d1c-8e34-b787de04b37c None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:14.525542 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:14.548824 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9399b4c4-cf5d-4366-8ed1-ce6872441d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:14.549379 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9399b4c4-cf5d-4366-8ed1-ce6872441d71 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:14.549653 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9399b4c4-cf5d-4366-8ed1-ce6872441d71 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:14.549949 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9399b4c4-cf5d-4366-8ed1-ce6872441d71 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:14.550517 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9399b4c4-cf5d-4366-8ed1-ce6872441d71 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:14.551863 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9399b4c4-cf5d-4366-8ed1-ce6872441d71 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:14.561257 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-9399b4c4-cf5d-4366-8ed1-ce6872441d71 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 28 15:32:14.561257 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:32:14.561257 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:32:14.561257 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:32:14.561257 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:32:14.561257 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:32:14.561257 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:32:14.561257 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:32:14.561257 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 28 15:32:14.561257 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:32:14.561257 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:32:14.561257 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:32:14.561257 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:32:14.561257 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:32:14.561257 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:32:14.561257 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:32:14.561257 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:32:14.562506 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:32:14.562506 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:32:14.562506 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:32:14.562506 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:32:14.562506 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:32:14.562506 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:32:14.562506 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:32:14.562506 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:32:14.562506 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:32:14.562506 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:32:14.562506 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:32:14.562506 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:32:14.562506 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:32:14.562506 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:32:14.562506 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:32:14.562506 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:32:14.563382 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:32:14.563382 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:32:14.563382 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:32:14.563382 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:32:14.563382 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:32:14.563382 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:32:14.563382 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:32:14.563382 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:32:14.563382 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:32:14.563382 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:32:14.563382 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:32:14.563382 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:32:14.563382 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 28 15:32:14.563382 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:32:14.563382 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 642/1290] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Sep 28 15:32:14 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:32:14.568795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec948d4-edea-4171-a12f-b9d9941090dd None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:14.663012 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec948d4-edea-4171-a12f-b9d9941090dd None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:14.694944 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eed387-68f9-45dd-ae36-cf80f05989bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:14.695536 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eed387-68f9-45dd-ae36-cf80f05989bd None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:14.695810 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eed387-68f9-45dd-ae36-cf80f05989bd None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:14.696078 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eed387-68f9-45dd-ae36-cf80f05989bd None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:14.696685 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eed387-68f9-45dd-ae36-cf80f05989bd None admin] RBAC: Adding query filter params (name=placement) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:32:14.696979 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eed387-68f9-45dd-ae36-cf80f05989bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:14.698411 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eed387-68f9-45dd-ae36-cf80f05989bd None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:14.702969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 649/1291] 10.222.0.52 () {56 vars in 1116 bytes} [Wed Sep 28 15:32:14 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:32:14.708641 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9399b4c4-cf5d-4366-8ed1-ce6872441d71 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:14.804331 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9399b4c4-cf5d-4366-8ed1-ce6872441d71 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:14.835548 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3b407e-cf07-40dd-8cd3-b7678e7c28b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:14.836123 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b407e-cf07-40dd-8cd3-b7678e7c28b4 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:14.836396 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b407e-cf07-40dd-8cd3-b7678e7c28b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:14.836661 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b407e-cf07-40dd-8cd3-b7678e7c28b4 None admin] PATH_INFO: `/v3/services` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:14.837265 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3b407e-cf07-40dd-8cd3-b7678e7c28b4 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:32:14.837560 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3b407e-cf07-40dd-8cd3-b7678e7c28b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:14.839435 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3b407e-cf07-40dd-8cd3-b7678e7c28b4 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:14.843595 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 643/1292] 10.222.0.52 () {56 vars in 1116 bytes} [Wed Sep 28 15:32:14 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:32:16.946459 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1ab529-7dd0-46c3-b2da-2455d87883f5 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:16.946984 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1ab529-7dd0-46c3-b2da-2455d87883f5 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:16.946984 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1ab529-7dd0-46c3-b2da-2455d87883f5 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:16.948939 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 650/1293] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:32:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:32:16.954176 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8178d0-28fe-4df1-86bf-8eb8392c059f None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:16.954581 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8178d0-28fe-4df1-86bf-8eb8392c059f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:16.954718 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8178d0-28fe-4df1-86bf-8eb8392c059f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:16.972317 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-5b8178d0-28fe-4df1-86bf-8eb8392c059f None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:32:16.973482 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-5b8178d0-28fe-4df1-86bf-8eb8392c059f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:32:17.002341 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:17.038351 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 644/1294] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:32:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:32:17.305826 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1ab529-7dd0-46c3-b2da-2455d87883f5 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:17.336278 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:17.404682 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1ab529-7dd0-46c3-b2da-2455d87883f5 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:17.436902 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e6f91a-6d9c-4c5c-806f-c13e43d520fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:17.437483 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6f91a-6d9c-4c5c-806f-c13e43d520fd None admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:17.437759 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6f91a-6d9c-4c5c-806f-c13e43d520fd None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:17.438061 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6f91a-6d9c-4c5c-806f-c13e43d520fd None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:17.438700 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e6f91a-6d9c-4c5c-806f-c13e43d520fd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:17.440043 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e6f91a-6d9c-4c5c-806f-c13e43d520fd None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:17.453642 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 651/1295] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Sep 28 15:32:17 2022] POST /identity/v3/services => generated 246 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:32:19.583556 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c4616-6b2e-4928-b99f-ef20794ff184 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:19.584096 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c4616-6b2e-4928-b99f-ef20794ff184 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:19.584096 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c4616-6b2e-4928-b99f-ef20794ff184 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:19.585415 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 645/1296] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:32:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:32:19.590638 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f20db1-65d9-4c03-9c27-011795de16fa None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:19.591044 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f20db1-65d9-4c03-9c27-011795de16fa None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:19.591185 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f20db1-65d9-4c03-9c27-011795de16fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:19.609346 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-17f20db1-65d9-4c03-9c27-011795de16fa None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:32:19.610566 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-17f20db1-65d9-4c03-9c27-011795de16fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:32:19.680099 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 652/1297] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:32:19 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:32:19.947633 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154c4616-6b2e-4928-b99f-ef20794ff184 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:20.055002 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154c4616-6b2e-4928-b99f-ef20794ff184 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:20.090037 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5ff0a0-624c-4f4d-8e4f-ea85d4cdfa79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:20.090661 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ff0a0-624c-4f4d-8e4f-ea85d4cdfa79 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:20.090938 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ff0a0-624c-4f4d-8e4f-ea85d4cdfa79 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:20.091205 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ff0a0-624c-4f4d-8e4f-ea85d4cdfa79 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:20.091777 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5ff0a0-624c-4f4d-8e4f-ea85d4cdfa79 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:20.093137 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5ff0a0-624c-4f4d-8e4f-ea85d4cdfa79 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:20.101654 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-4c5ff0a0-624c-4f4d-8e4f-ea85d4cdfa79 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 28 15:32:20.101654 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:32:20.101654 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:32:20.101654 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:32:20.101654 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:32:20.101654 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:32:20.101654 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:32:20.101654 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:32:20.101654 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 28 15:32:20.101654 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:32:20.101654 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:32:20.101654 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:32:20.101654 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:32:20.101654 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:32:20.101654 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:32:20.101654 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:32:20.101654 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:32:20.102838 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:32:20.102838 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:32:20.102838 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:32:20.102838 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:32:20.102838 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:32:20.102838 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:32:20.102838 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:32:20.102838 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:32:20.102838 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:32:20.102838 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:32:20.102838 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:32:20.102838 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:32:20.102838 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:32:20.102838 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:32:20.102838 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:32:20.102838 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:32:20.104290 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:32:20.104290 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:32:20.104290 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:32:20.104290 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:32:20.104290 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:32:20.104290 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:32:20.104290 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:32:20.104290 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:32:20.104290 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:32:20.104290 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:32:20.104290 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:32:20.104290 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:32:20.104290 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 28 15:32:20.104290 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:32:20.104290 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 646/1298] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Sep 28 15:32:19 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:32:20.109543 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f20db1-65d9-4c03-9c27-011795de16fa None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:20.127514 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:20.205888 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f20db1-65d9-4c03-9c27-011795de16fa None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:20.238433 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97989c5-8f03-427b-9e6a-f94fd2173d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:20.239017 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97989c5-8f03-427b-9e6a-f94fd2173d88 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:20.239293 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97989c5-8f03-427b-9e6a-f94fd2173d88 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:20.239561 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97989c5-8f03-427b-9e6a-f94fd2173d88 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:20.240171 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97989c5-8f03-427b-9e6a-f94fd2173d88 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:32:20.240466 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97989c5-8f03-427b-9e6a-f94fd2173d88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:20.241809 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97989c5-8f03-427b-9e6a-f94fd2173d88 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:20.246767 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 653/1299] 10.222.0.52 () {56 vars in 1116 bytes} [Wed Sep 28 15:32:20 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:32:20.252867 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5ff0a0-624c-4f4d-8e4f-ea85d4cdfa79 None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:20.284067 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:20.350205 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5ff0a0-624c-4f4d-8e4f-ea85d4cdfa79 None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:20.382043 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f44d6e-7a0a-4ac6-ba87-1ae545585b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:20.382636 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f44d6e-7a0a-4ac6-ba87-1ae545585b55 None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:20.382912 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f44d6e-7a0a-4ac6-ba87-1ae545585b55 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:20.383190 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f44d6e-7a0a-4ac6-ba87-1ae545585b55 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:20.383926 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f44d6e-7a0a-4ac6-ba87-1ae545585b55 None admin] RBAC: Adding query filter params (interface=public, service_id=74d12783530a400ebcf2bde8f85ed93a, region_id=RegionOne) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:32:20.384233 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f44d6e-7a0a-4ac6-ba87-1ae545585b55 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:20.386118 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f44d6e-7a0a-4ac6-ba87-1ae545585b55 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:20.390896 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 647/1300] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Sep 28 15:32:20 2022] GET /identity/v3/endpoints?service_id=74d12783530a400ebcf2bde8f85ed93a&interface=public®ion_id=RegionOne => generated 194 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:32:20.396468 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97989c5-8f03-427b-9e6a-f94fd2173d88 None admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:20.473627 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:20.491039 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97989c5-8f03-427b-9e6a-f94fd2173d88 None admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:20.523872 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee9b917-8127-4ab1-9cc0-1eae012f3046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:20.524457 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9b917-8127-4ab1-9cc0-1eae012f3046 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:20.524731 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9b917-8127-4ab1-9cc0-1eae012f3046 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:20.525595 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9b917-8127-4ab1-9cc0-1eae012f3046 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:20.526206 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee9b917-8127-4ab1-9cc0-1eae012f3046 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:20.527557 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee9b917-8127-4ab1-9cc0-1eae012f3046 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:20.534023 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 654/1301] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Sep 28 15:32:20 2022] GET /identity/v3/services => generated 2191 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:32:22.648372 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa21391-0886-4931-99aa-dd2b67ac6890 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:22.648935 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa21391-0886-4931-99aa-dd2b67ac6890 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:22.648935 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa21391-0886-4931-99aa-dd2b67ac6890 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:22.650301 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 648/1302] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:32:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:32:22.655440 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f20df40-24f1-42a1-add7-6ac3a6f4f02b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:22.655865 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f20df40-24f1-42a1-add7-6ac3a6f4f02b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:22.655989 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f20df40-24f1-42a1-add7-6ac3a6f4f02b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:22.674266 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-8f20df40-24f1-42a1-add7-6ac3a6f4f02b None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:32:22.675431 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8f20df40-24f1-42a1-add7-6ac3a6f4f02b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:32:22.738942 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 655/1303] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:32:22 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 28 15:32:23.012834 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa21391-0886-4931-99aa-dd2b67ac6890 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:23.113351 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:23.120738 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa21391-0886-4931-99aa-dd2b67ac6890 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:23.153538 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cc63e2-0d04-4082-ae4e-2ddbd059905e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:23.154186 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc63e2-0d04-4082-ae4e-2ddbd059905e None admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:23.154460 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc63e2-0d04-4082-ae4e-2ddbd059905e None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:23.154726 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc63e2-0d04-4082-ae4e-2ddbd059905e None admin] PATH_INFO: `/v3/services/placement` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:23.155294 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cc63e2-0d04-4082-ae4e-2ddbd059905e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:23.156652 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cc63e2-0d04-4082-ae4e-2ddbd059905e None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:23.165121 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application [None req-f1cc63e2-0d04-4082-ae4e-2ddbd059905e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 28 15:32:23.165121 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:32:23.165121 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 28 15:32:23.165121 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 28 15:32:23.165121 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 28 15:32:23.165121 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 28 15:32:23.165121 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 28 15:32:23.165121 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:32:23.165121 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 28 15:32:23.165121 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:32:23.165121 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 28 15:32:23.165121 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:32:23.165121 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 28 15:32:23.165121 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 28 15:32:23.165121 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 28 15:32:23.165121 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 28 15:32:23.165121 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 28 15:32:23.166712 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 28 15:32:23.166712 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 28 15:32:23.166712 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 28 15:32:23.166712 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 28 15:32:23.166712 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 28 15:32:23.166712 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 28 15:32:23.166712 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 28 15:32:23.166712 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 28 15:32:23.166712 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 28 15:32:23.166712 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 28 15:32:23.166712 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 28 15:32:23.166712 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 28 15:32:23.166712 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 28 15:32:23.166712 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 28 15:32:23.166712 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 28 15:32:23.166712 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.get_or_create( Sep 28 15:32:23.167671 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 28 15:32:23.167671 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application with Lock( Sep 28 15:32:23.167671 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 28 15:32:23.167671 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self._enter() Sep 28 15:32:23.167671 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 28 15:32:23.167671 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 28 15:32:23.167671 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 28 15:32:23.167671 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application return self.creator() Sep 28 15:32:23.167671 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 28 15:32:23.167671 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application created_value = creator( Sep 28 15:32:23.167671 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 28 15:32:23.167671 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 28 15:32:23.167671 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 28 15:32:23.167671 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: ERROR keystone.server.flask.application Sep 28 15:32:23.167671 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 649/1304] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Sep 28 15:32:23 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 28 15:32:23.172896 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f20df40-24f1-42a1-add7-6ac3a6f4f02b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:23.249394 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:23.272436 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f20df40-24f1-42a1-add7-6ac3a6f4f02b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:23.305553 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce1415-f09d-4914-96de-d0c6cfd58221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:23.306204 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce1415-f09d-4914-96de-d0c6cfd58221 None admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:23.306484 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce1415-f09d-4914-96de-d0c6cfd58221 None admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:23.306756 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce1415-f09d-4914-96de-d0c6cfd58221 None admin] PATH_INFO: `/v3/services` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:23.307373 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ce1415-f09d-4914-96de-d0c6cfd58221 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:32:23.307670 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ce1415-f09d-4914-96de-d0c6cfd58221 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:23.309038 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ce1415-f09d-4914-96de-d0c6cfd58221 None admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:23.313945 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 656/1305] 10.222.0.52 () {56 vars in 1116 bytes} [Wed Sep 28 15:32:23 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:32:23.319646 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cc63e2-0d04-4082-ae4e-2ddbd059905e None admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:32:23.416106 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cc63e2-0d04-4082-ae4e-2ddbd059905e None admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:32:23.448721 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050e4dcc-72b2-40bb-a6e0-9a1c434ba113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:32:23.449311 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e4dcc-72b2-40bb-a6e0-9a1c434ba113 None admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:32:23.449587 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e4dcc-72b2-40bb-a6e0-9a1c434ba113 None admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:32:23.449881 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e4dcc-72b2-40bb-a6e0-9a1c434ba113 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:32:23.450531 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050e4dcc-72b2-40bb-a6e0-9a1c434ba113 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:32:23.451885 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050e4dcc-72b2-40bb-a6e0-9a1c434ba113 None admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:32:23.471219 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 650/1306] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:32:23 2022] POST /identity/v3/endpoints => generated 331 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:32:33.311045 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=30 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:32:33.466908 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:27.670465 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756dc38-054a-4c83-ac65-2386cea67020 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:27.671053 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756dc38-054a-4c83-ac65-2386cea67020 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:27.671053 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756dc38-054a-4c83-ac65-2386cea67020 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:27.673268 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 657/1307] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:33:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:33:27.679129 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecdbcf3-c9b9-4b72-b3cc-5be0cb21af22 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:27.679588 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecdbcf3-c9b9-4b72-b3cc-5be0cb21af22 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:27.679681 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecdbcf3-c9b9-4b72-b3cc-5be0cb21af22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:27.838753 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-3ecdbcf3-c9b9-4b72-b3cc-5be0cb21af22 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:33:27.839974 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3ecdbcf3-c9b9-4b72-b3cc-5be0cb21af22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:33:28.085880 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 651/1308] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:33:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:33:28.391730 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b93ecc-e2d7-4e98-959d-1097c61cab53 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:28.392074 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b93ecc-e2d7-4e98-959d-1097c61cab53 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:28.392335 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b93ecc-e2d7-4e98-959d-1097c61cab53 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:28.393985 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 658/1309] 10.222.0.52 () {54 vars in 964 bytes} [Wed Sep 28 15:33:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:33:28.400807 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481bffde-f800-4e3d-8329-d2d2e4ff0163 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:28.401085 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481bffde-f800-4e3d-8329-d2d2e4ff0163 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:28.401360 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481bffde-f800-4e3d-8329-d2d2e4ff0163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:28.502413 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-481bffde-f800-4e3d-8329-d2d2e4ff0163 None None] MFA Rules not processed for user `b13b0533444a449ead00a389c5f6bd1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:33:28.504095 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-481bffde-f800-4e3d-8329-d2d2e4ff0163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:33:28.528226 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:28.596658 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 652/1310] 10.222.0.52 () {58 vars in 1061 bytes} [Wed Sep 28 15:33:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:33:28.606002 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b93ecc-e2d7-4e98-959d-1097c61cab53 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:28.607770 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-78b93ecc-e2d7-4e98-959d-1097c61cab53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:33:28.729585 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b93ecc-e2d7-4e98-959d-1097c61cab53 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:28.818491 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d6fdf7-03e7-4aef-9ea4-9988387b8230 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:28.819283 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d6fdf7-03e7-4aef-9ea4-9988387b8230 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:28.819656 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d6fdf7-03e7-4aef-9ea4-9988387b8230 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:28.820024 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d6fdf7-03e7-4aef-9ea4-9988387b8230 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:28.821724 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-86d6fdf7-03e7-4aef-9ea4-9988387b8230 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:33:28.880874 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:28.941514 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d6fdf7-03e7-4aef-9ea4-9988387b8230 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:28.943135 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d6fdf7-03e7-4aef-9ea4-9988387b8230 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:28.965604 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 659/1311] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:33:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:33:31.449809 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa980a-b6aa-4a01-a8cf-3ef35b8b1aa4 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:31.450363 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa980a-b6aa-4a01-a8cf-3ef35b8b1aa4 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:31.450363 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa980a-b6aa-4a01-a8cf-3ef35b8b1aa4 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:31.451705 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 653/1312] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:33:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:33:31.457511 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f2ed16-d25b-4a6a-9d35-5153d9d57897 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:31.457926 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f2ed16-d25b-4a6a-9d35-5153d9d57897 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:31.458084 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f2ed16-d25b-4a6a-9d35-5153d9d57897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:31.527518 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-42f2ed16-d25b-4a6a-9d35-5153d9d57897 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:33:31.528707 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-42f2ed16-d25b-4a6a-9d35-5153d9d57897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:33:31.622468 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 660/1313] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:33:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:33:31.907667 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfa980a-b6aa-4a01-a8cf-3ef35b8b1aa4 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:31.909253 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ebfa980a-b6aa-4a01-a8cf-3ef35b8b1aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:33:32.011385 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfa980a-b6aa-4a01-a8cf-3ef35b8b1aa4 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:32.084449 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:32.086775 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a32a7-53b4-4462-a7fc-71c822c727f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:32.087596 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a32a7-53b4-4462-a7fc-71c822c727f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:32.088055 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a32a7-53b4-4462-a7fc-71c822c727f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:32.088343 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a32a7-53b4-4462-a7fc-71c822c727f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:32.090138 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-468a32a7-53b4-4462-a7fc-71c822c727f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:33:32.197157 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468a32a7-53b4-4462-a7fc-71c822c727f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:32.198534 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468a32a7-53b4-4462-a7fc-71c822c727f9 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:32.216282 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 654/1314] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:33:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:33:34.636656 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03878770-37bb-4168-9cef-caefd1fa2dde None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:34.637213 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03878770-37bb-4168-9cef-caefd1fa2dde None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:34.637213 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03878770-37bb-4168-9cef-caefd1fa2dde None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:34.638585 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 661/1315] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:33:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:33:34.643794 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033bb4e-bdd2-454e-9d1e-7edfd00e2941 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:34.644200 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033bb4e-bdd2-454e-9d1e-7edfd00e2941 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:34.644362 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033bb4e-bdd2-454e-9d1e-7edfd00e2941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:34.713590 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-a033bb4e-bdd2-454e-9d1e-7edfd00e2941 None None] MFA Rules not processed for user `b0d4ab9befe041b08d614daa5c9e423c`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:33:34.714863 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a033bb4e-bdd2-454e-9d1e-7edfd00e2941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:33:34.767231 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:34.811480 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 655/1316] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:33:34 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:33:35.107104 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03878770-37bb-4168-9cef-caefd1fa2dde None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:35.108625 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-03878770-37bb-4168-9cef-caefd1fa2dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:33:35.118188 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:35.210338 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03878770-37bb-4168-9cef-caefd1fa2dde None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:35.281345 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4206a195-448e-4f7a-a320-ee085c572d8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:35.281978 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4206a195-448e-4f7a-a320-ee085c572d8e service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:35.282258 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4206a195-448e-4f7a-a320-ee085c572d8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:35.282525 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4206a195-448e-4f7a-a320-ee085c572d8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:35.284200 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4206a195-448e-4f7a-a320-ee085c572d8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:33:35.345214 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:35.390827 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4206a195-448e-4f7a-a320-ee085c572d8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:35.392197 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4206a195-448e-4f7a-a320-ee085c572d8e service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:35.411863 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 662/1317] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:33:35 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:33:38.062151 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60be64-df1d-4932-b09b-80a562063aba None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:38.062719 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60be64-df1d-4932-b09b-80a562063aba None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:38.062719 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60be64-df1d-4932-b09b-80a562063aba None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:38.064026 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 656/1318] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:33:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:33:38.069924 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e421a8f-6fe9-4945-a7c1-decc287cb542 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:38.070321 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e421a8f-6fe9-4945-a7c1-decc287cb542 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:38.070480 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e421a8f-6fe9-4945-a7c1-decc287cb542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:38.138885 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-5e421a8f-6fe9-4945-a7c1-decc287cb542 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:33:38.140077 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5e421a8f-6fe9-4945-a7c1-decc287cb542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:33:38.234355 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 663/1319] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:33:38 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:33:38.516325 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e60be64-df1d-4932-b09b-80a562063aba None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:38.517887 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-0e60be64-df1d-4932-b09b-80a562063aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:33:38.618585 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e60be64-df1d-4932-b09b-80a562063aba None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:38.691985 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847da29b-bdf3-430d-b1e3-362110fdb48f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:38.692586 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847da29b-bdf3-430d-b1e3-362110fdb48f service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:38.692861 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847da29b-bdf3-430d-b1e3-362110fdb48f service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:38.693129 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847da29b-bdf3-430d-b1e3-362110fdb48f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:38.694738 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-847da29b-bdf3-430d-b1e3-362110fdb48f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:33:38.701495 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:38.801273 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847da29b-bdf3-430d-b1e3-362110fdb48f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:38.802735 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847da29b-bdf3-430d-b1e3-362110fdb48f service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:38.820299 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 657/1320] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:33:38 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:33:41.588210 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73bbc1c-6587-4763-bee2-bf0856bdd862 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:41.588780 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73bbc1c-6587-4763-bee2-bf0856bdd862 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:41.588780 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73bbc1c-6587-4763-bee2-bf0856bdd862 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:41.590125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 664/1321] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:33:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:33:41.595260 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d7160-f14e-4206-8985-74125fefde68 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:41.595664 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d7160-f14e-4206-8985-74125fefde68 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:41.595798 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d7160-f14e-4206-8985-74125fefde68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:41.667225 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-9f9d7160-f14e-4206-8985-74125fefde68 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:33:41.668460 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-9f9d7160-f14e-4206-8985-74125fefde68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:33:41.730865 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:41.773680 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 658/1322] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:33:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:33:42.059233 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73bbc1c-6587-4763-bee2-bf0856bdd862 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:42.060743 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e73bbc1c-6587-4763-bee2-bf0856bdd862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:33:42.079976 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:42.163524 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73bbc1c-6587-4763-bee2-bf0856bdd862 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:42.236327 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f183a56-3ecd-4ca0-a8bb-ac7272abe47c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:42.237474 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f183a56-3ecd-4ca0-a8bb-ac7272abe47c service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:42.237754 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f183a56-3ecd-4ca0-a8bb-ac7272abe47c service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:42.238051 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f183a56-3ecd-4ca0-a8bb-ac7272abe47c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:42.239642 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8f183a56-3ecd-4ca0-a8bb-ac7272abe47c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:33:42.314193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:42.352049 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f183a56-3ecd-4ca0-a8bb-ac7272abe47c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:42.353487 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f183a56-3ecd-4ca0-a8bb-ac7272abe47c service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:42.371187 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 665/1323] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:33:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:33:44.569321 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092c813-f1d6-4eee-aac0-d4c167198d16 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:44.569880 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092c813-f1d6-4eee-aac0-d4c167198d16 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:44.569880 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092c813-f1d6-4eee-aac0-d4c167198d16 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:44.571233 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 659/1324] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:33:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:33:44.576573 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2248a32-de29-4e78-a8a0-83b80b02e011 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:44.577000 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2248a32-de29-4e78-a8a0-83b80b02e011 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:44.577265 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2248a32-de29-4e78-a8a0-83b80b02e011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:44.643775 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d2248a32-de29-4e78-a8a0-83b80b02e011 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:33:44.645270 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d2248a32-de29-4e78-a8a0-83b80b02e011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:33:44.750510 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 666/1325] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:33:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:33:45.035867 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0092c813-f1d6-4eee-aac0-d4c167198d16 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:45.037425 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-0092c813-f1d6-4eee-aac0-d4c167198d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:33:45.146318 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0092c813-f1d6-4eee-aac0-d4c167198d16 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:45.226637 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a9e708-dd3c-40f8-a92a-1e59e5a2fb22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:45.227244 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9e708-dd3c-40f8-a92a-1e59e5a2fb22 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:45.227517 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9e708-dd3c-40f8-a92a-1e59e5a2fb22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:45.227785 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a9e708-dd3c-40f8-a92a-1e59e5a2fb22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:45.229347 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a4a9e708-dd3c-40f8-a92a-1e59e5a2fb22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:33:45.246101 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:45.341364 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a9e708-dd3c-40f8-a92a-1e59e5a2fb22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:45.342817 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a9e708-dd3c-40f8-a92a-1e59e5a2fb22 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:45.360377 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 660/1326] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:33:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:33:47.589283 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb4926-cacc-4439-a446-e56d7a626e5c None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:47.589795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb4926-cacc-4439-a446-e56d7a626e5c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:47.589795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb4926-cacc-4439-a446-e56d7a626e5c None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:47.591154 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 667/1327] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:33:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:33:47.597039 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7305f392-6e2a-4145-871f-2913418849fb None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:47.597447 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7305f392-6e2a-4145-871f-2913418849fb None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:47.597617 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7305f392-6e2a-4145-871f-2913418849fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:47.663581 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-7305f392-6e2a-4145-871f-2913418849fb None None] MFA Rules not processed for user `b0d4ab9befe041b08d614daa5c9e423c`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:33:47.664762 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7305f392-6e2a-4145-871f-2913418849fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:33:47.731638 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:47.764652 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 661/1328] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:33:47 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:33:48.048849 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eb4926-cacc-4439-a446-e56d7a626e5c None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:48.050421 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-51eb4926-cacc-4439-a446-e56d7a626e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:33:48.083931 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:48.159640 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eb4926-cacc-4439-a446-e56d7a626e5c None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:48.236735 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530d17af-d401-408f-9b6d-5a3713717060 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:48.237256 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d17af-d401-408f-9b6d-5a3713717060 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:48.237533 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d17af-d401-408f-9b6d-5a3713717060 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:48.237797 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d17af-d401-408f-9b6d-5a3713717060 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:48.239397 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-530d17af-d401-408f-9b6d-5a3713717060 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:33:48.320216 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:48.352431 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530d17af-d401-408f-9b6d-5a3713717060 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:48.353876 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530d17af-d401-408f-9b6d-5a3713717060 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:48.372303 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 668/1329] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:33:48 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:33:50.750505 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8a306-facb-4830-906b-339a8ef5738d None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:50.751067 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8a306-facb-4830-906b-339a8ef5738d None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:50.751067 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8a306-facb-4830-906b-339a8ef5738d None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:50.753016 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 662/1330] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:33:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:33:50.758193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1919c-277e-48a1-858e-734dc0a58ef1 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:50.758605 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1919c-277e-48a1-858e-734dc0a58ef1 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:50.758735 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1919c-277e-48a1-858e-734dc0a58ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:50.824527 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-44f1919c-277e-48a1-858e-734dc0a58ef1 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:33:50.825716 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-44f1919c-277e-48a1-858e-734dc0a58ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:33:50.918332 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 669/1331] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:33:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:33:51.196166 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d8a306-facb-4830-906b-339a8ef5738d None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:51.197739 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d2d8a306-facb-4830-906b-339a8ef5738d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:33:51.300799 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d8a306-facb-4830-906b-339a8ef5738d None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:51.374001 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4960da-3497-41bb-aea8-60c6bd53f31a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:51.374599 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4960da-3497-41bb-aea8-60c6bd53f31a service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:51.374871 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4960da-3497-41bb-aea8-60c6bd53f31a service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:51.375136 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4960da-3497-41bb-aea8-60c6bd53f31a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:51.376775 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4e4960da-3497-41bb-aea8-60c6bd53f31a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:33:51.403661 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:51.483730 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4960da-3497-41bb-aea8-60c6bd53f31a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:51.485072 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4960da-3497-41bb-aea8-60c6bd53f31a service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:51.502780 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 663/1332] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:33:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:33:53.715907 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92e199-4f2b-47a4-bf1c-8aff780a2c01 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:53.716449 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92e199-4f2b-47a4-bf1c-8aff780a2c01 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:53.716449 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92e199-4f2b-47a4-bf1c-8aff780a2c01 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:53.717807 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 670/1333] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:33:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:33:53.723636 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c21484-0c04-4296-957f-6ec5617ffce1 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:53.724040 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c21484-0c04-4296-957f-6ec5617ffce1 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:53.724180 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c21484-0c04-4296-957f-6ec5617ffce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:53.791279 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-17c21484-0c04-4296-957f-6ec5617ffce1 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:33:53.793296 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-17c21484-0c04-4296-957f-6ec5617ffce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:33:53.869019 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:53.890030 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 664/1334] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:33:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:33:54.184951 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b92e199-4f2b-47a4-bf1c-8aff780a2c01 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:54.186539 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1b92e199-4f2b-47a4-bf1c-8aff780a2c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:33:54.226256 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:54.288732 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b92e199-4f2b-47a4-bf1c-8aff780a2c01 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:54.363884 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1325b63-0637-41f7-85cf-3eca53ab1d1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:54.364491 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1325b63-0637-41f7-85cf-3eca53ab1d1b service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:54.364771 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1325b63-0637-41f7-85cf-3eca53ab1d1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:54.365042 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1325b63-0637-41f7-85cf-3eca53ab1d1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:54.366656 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f1325b63-0637-41f7-85cf-3eca53ab1d1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:33:54.448949 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:54.474536 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1325b63-0637-41f7-85cf-3eca53ab1d1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:54.475964 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1325b63-0637-41f7-85cf-3eca53ab1d1b service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:54.494008 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 671/1335] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:33:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:33:56.982840 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f96fb-a3a4-4c57-a914-c45d9338d7cc None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:56.982840 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f96fb-a3a4-4c57-a914-c45d9338d7cc None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:56.984425 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f96fb-a3a4-4c57-a914-c45d9338d7cc None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:56.985099 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 665/1336] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:33:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:33:56.991121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874581c6-924f-407a-a33d-bfbb2c2dbbe9 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:56.991594 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874581c6-924f-407a-a33d-bfbb2c2dbbe9 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:56.991708 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874581c6-924f-407a-a33d-bfbb2c2dbbe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:57.062918 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-874581c6-924f-407a-a33d-bfbb2c2dbbe9 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:33:57.064659 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-874581c6-924f-407a-a33d-bfbb2c2dbbe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:33:57.159821 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 672/1337] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:33:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:33:57.447156 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41f96fb-a3a4-4c57-a914-c45d9338d7cc None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:33:57.448693 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b41f96fb-a3a4-4c57-a914-c45d9338d7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:33:57.551139 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41f96fb-a3a4-4c57-a914-c45d9338d7cc None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:33:57.622777 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae33b09e-13e3-4f42-8465-cf4af64f273c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:33:57.623377 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae33b09e-13e3-4f42-8465-cf4af64f273c service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:33:57.623654 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae33b09e-13e3-4f42-8465-cf4af64f273c service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:33:57.623922 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae33b09e-13e3-4f42-8465-cf4af64f273c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:33:57.625498 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ae33b09e-13e3-4f42-8465-cf4af64f273c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:33:57.660823 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:33:57.732201 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae33b09e-13e3-4f42-8465-cf4af64f273c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:33:57.733615 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae33b09e-13e3-4f42-8465-cf4af64f273c service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:33:57.752428 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 666/1338] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:33:57 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:34:01.805476 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb8acf9-cd01-42f0-8c39-512561e674f7 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:01.806059 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb8acf9-cd01-42f0-8c39-512561e674f7 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:01.806059 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb8acf9-cd01-42f0-8c39-512561e674f7 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:01.807516 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 673/1339] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:34:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:34:01.812972 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39203af-a310-4912-bf23-76171c6a888e None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:01.812972 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39203af-a310-4912-bf23-76171c6a888e None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:01.813480 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39203af-a310-4912-bf23-76171c6a888e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:01.884248 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-a39203af-a310-4912-bf23-76171c6a888e None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:34:01.885516 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a39203af-a310-4912-bf23-76171c6a888e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:01.976230 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:01.989893 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 667/1340] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:34:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:34:02.291285 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05063f-78b0-4412-99dc-ed735971142b None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:02.291668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05063f-78b0-4412-99dc-ed735971142b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:02.292025 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05063f-78b0-4412-99dc-ed735971142b None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:02.293588 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 674/1341] 10.222.0.52 () {54 vars in 964 bytes} [Wed Sep 28 15:34:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:34:02.299946 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea38c2-35c4-4754-9e1b-e98f3f1a376c None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:02.300347 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea38c2-35c4-4754-9e1b-e98f3f1a376c None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:02.300690 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea38c2-35c4-4754-9e1b-e98f3f1a376c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:02.375652 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-baea38c2-35c4-4754-9e1b-e98f3f1a376c None None] MFA Rules not processed for user `b13b0533444a449ead00a389c5f6bd1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:34:02.376895 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-baea38c2-35c4-4754-9e1b-e98f3f1a376c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:02.450809 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 668/1342] 10.222.0.52 () {58 vars in 1061 bytes} [Wed Sep 28 15:34:02 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:34:02.459746 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d05063f-78b0-4412-99dc-ed735971142b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:34:02.461368 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9d05063f-78b0-4412-99dc-ed735971142b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:02.510688 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:02.574230 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d05063f-78b0-4412-99dc-ed735971142b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:34:02.649502 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4271e20-3ce1-480a-bf53-cfc87d7efb2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:34:02.650133 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4271e20-3ce1-480a-bf53-cfc87d7efb2b service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:02.650427 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4271e20-3ce1-480a-bf53-cfc87d7efb2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:02.650680 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4271e20-3ce1-480a-bf53-cfc87d7efb2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:02.652243 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c4271e20-3ce1-480a-bf53-cfc87d7efb2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:02.752346 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:02.771121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4271e20-3ce1-480a-bf53-cfc87d7efb2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:34:02.772575 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4271e20-3ce1-480a-bf53-cfc87d7efb2b service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:34:02.792427 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 675/1343] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:34:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:34:06.241580 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e748a11-cb7c-4453-b78a-8cf1ac3e537f None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:06.242110 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e748a11-cb7c-4453-b78a-8cf1ac3e537f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:06.242201 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e748a11-cb7c-4453-b78a-8cf1ac3e537f None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:06.243871 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 669/1344] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:34:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:34:06.248810 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f33f1f-ec62-4ef8-b9d4-cc5198d28ce1 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:06.249332 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f33f1f-ec62-4ef8-b9d4-cc5198d28ce1 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:06.249332 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f33f1f-ec62-4ef8-b9d4-cc5198d28ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:06.345497 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-20f33f1f-ec62-4ef8-b9d4-cc5198d28ce1 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:34:06.346743 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-20f33f1f-ec62-4ef8-b9d4-cc5198d28ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:06.448339 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 676/1345] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:34:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:34:06.733906 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e748a11-cb7c-4453-b78a-8cf1ac3e537f None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:34:06.736086 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-0e748a11-cb7c-4453-b78a-8cf1ac3e537f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:06.823306 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:06.840307 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e748a11-cb7c-4453-b78a-8cf1ac3e537f None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:34:06.914039 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95ff266-7e4f-4dd8-8c31-2ea7bb9404db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:34:06.914647 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ff266-7e4f-4dd8-8c31-2ea7bb9404db service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:06.914923 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ff266-7e4f-4dd8-8c31-2ea7bb9404db service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:06.915189 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ff266-7e4f-4dd8-8c31-2ea7bb9404db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:06.916772 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e95ff266-7e4f-4dd8-8c31-2ea7bb9404db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:07.028708 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95ff266-7e4f-4dd8-8c31-2ea7bb9404db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:34:07.030093 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95ff266-7e4f-4dd8-8c31-2ea7bb9404db service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:34:07.049085 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 670/1346] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:34:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:34:11.336093 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ed0dfa-8b7e-4c24-861e-2d1391ae84bb None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:11.336538 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ed0dfa-8b7e-4c24-861e-2d1391ae84bb None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:11.336626 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ed0dfa-8b7e-4c24-861e-2d1391ae84bb None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:11.340117 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 677/1347] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:34:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:34:11.344344 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acbd623-913f-4113-8356-1b68f0cb059a None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:11.344823 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acbd623-913f-4113-8356-1b68f0cb059a None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:11.344918 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acbd623-913f-4113-8356-1b68f0cb059a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:11.363023 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:11.419919 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-4acbd623-913f-4113-8356-1b68f0cb059a None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:34:11.421132 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4acbd623-913f-4113-8356-1b68f0cb059a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:11.518160 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 671/1348] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:34:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:34:11.803035 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ed0dfa-8b7e-4c24-861e-2d1391ae84bb None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:34:11.804574 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-46ed0dfa-8b7e-4c24-861e-2d1391ae84bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:11.861529 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:11.907606 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ed0dfa-8b7e-4c24-861e-2d1391ae84bb None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:34:11.980409 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dc432a-cdf5-4afd-b5cc-2e0570deeaaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:34:11.981010 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc432a-cdf5-4afd-b5cc-2e0570deeaaf service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:11.981288 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc432a-cdf5-4afd-b5cc-2e0570deeaaf service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:11.981557 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc432a-cdf5-4afd-b5cc-2e0570deeaaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:11.983249 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-80dc432a-cdf5-4afd-b5cc-2e0570deeaaf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:12.088524 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:12.093040 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dc432a-cdf5-4afd-b5cc-2e0570deeaaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:34:12.095106 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dc432a-cdf5-4afd-b5cc-2e0570deeaaf service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:34:12.113955 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 678/1349] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:34:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:34:17.054936 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc796550-a82f-469f-a51d-cef7a24d42c9 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:17.055429 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc796550-a82f-469f-a51d-cef7a24d42c9 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:17.055509 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc796550-a82f-469f-a51d-cef7a24d42c9 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:17.057168 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 672/1350] 10.222.0.52 () {54 vars in 921 bytes} [Wed Sep 28 15:34:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:34:17.063049 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971d5acd-88e8-4d30-8665-304f08176374 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:17.063049 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971d5acd-88e8-4d30-8665-304f08176374 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:17.063516 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971d5acd-88e8-4d30-8665-304f08176374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:17.173781 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-971d5acd-88e8-4d30-8665-304f08176374 None None] MFA Rules not processed for user `de934d0105bd4d01ba6c06a4a0d36704`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:34:17.175011 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-971d5acd-88e8-4d30-8665-304f08176374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:17.337103 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 679/1351] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Sep 28 15:34:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:34:21.516680 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=59 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:21.603376 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb86d6-27f4-4ba3-954b-924b959a1606 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:21.603760 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb86d6-27f4-4ba3-954b-924b959a1606 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:21.603988 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb86d6-27f4-4ba3-954b-924b959a1606 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:21.605487 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 673/1352] 10.222.0.52 () {54 vars in 921 bytes} [Wed Sep 28 15:34:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:34:21.610951 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f92a46-59e0-45cd-99f9-c764f4100c9e None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:21.611830 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f92a46-59e0-45cd-99f9-c764f4100c9e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:21.612337 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f92a46-59e0-45cd-99f9-c764f4100c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:21.677047 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:21.705881 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-b4f92a46-59e0-45cd-99f9-c764f4100c9e None None] MFA Rules not processed for user `de934d0105bd4d01ba6c06a4a0d36704`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:34:21.707442 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b4f92a46-59e0-45cd-99f9-c764f4100c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:21.828807 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 680/1353] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Sep 28 15:34:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:34:23.210480 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e015e7-8a1d-4444-a831-0ccc135c6568 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:23.211045 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e015e7-8a1d-4444-a831-0ccc135c6568 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:23.211045 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e015e7-8a1d-4444-a831-0ccc135c6568 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:23.212375 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 674/1354] 10.222.0.52 () {54 vars in 921 bytes} [Wed Sep 28 15:34:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:34:23.217351 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c1072d-7d58-4b78-935e-a59f039e15d0 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:23.217782 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c1072d-7d58-4b78-935e-a59f039e15d0 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:23.217963 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c1072d-7d58-4b78-935e-a59f039e15d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:23.296356 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-24c1072d-7d58-4b78-935e-a59f039e15d0 None None] MFA Rules not processed for user `de934d0105bd4d01ba6c06a4a0d36704`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:34:23.297599 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-24c1072d-7d58-4b78-935e-a59f039e15d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:23.345353 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:23.411927 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 681/1355] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Sep 28 15:34:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:34:26.833821 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cdd349-7d23-4d8e-b498-4e90aba08a17 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:26.834380 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cdd349-7d23-4d8e-b498-4e90aba08a17 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:26.834380 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cdd349-7d23-4d8e-b498-4e90aba08a17 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:26.835687 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 675/1356] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:34:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:34:26.841092 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b7c5a-80a3-407a-9b07-06e134c15f1d None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:26.841580 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b7c5a-80a3-407a-9b07-06e134c15f1d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:26.841660 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b7c5a-80a3-407a-9b07-06e134c15f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:26.916181 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-966b7c5a-80a3-407a-9b07-06e134c15f1d None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:34:26.917293 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-966b7c5a-80a3-407a-9b07-06e134c15f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:27.019575 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 682/1357] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:34:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:34:27.316473 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31990a3d-cfe7-473e-bf64-c52fd954c953 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:27.317209 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31990a3d-cfe7-473e-bf64-c52fd954c953 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:27.317787 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31990a3d-cfe7-473e-bf64-c52fd954c953 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:27.319411 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 676/1358] 10.222.0.52 () {54 vars in 961 bytes} [Wed Sep 28 15:34:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:34:27.338651 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a3d35-d7a6-4e75-8b40-9824fed178b4 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:27.338913 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a3d35-d7a6-4e75-8b40-9824fed178b4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:27.339187 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a3d35-d7a6-4e75-8b40-9824fed178b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:27.360193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:27.433496 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-f42a3d35-d7a6-4e75-8b40-9824fed178b4 None None] MFA Rules not processed for user `8daf14b64bf04b2ba74d2fe1fd41ff71`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:34:27.434756 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f42a3d35-d7a6-4e75-8b40-9824fed178b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:27.568003 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 683/1359] 10.222.0.52 () {58 vars in 1058 bytes} [Wed Sep 28 15:34:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:34:27.575936 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31990a3d-cfe7-473e-bf64-c52fd954c953 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:34:27.578052 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-31990a3d-cfe7-473e-bf64-c52fd954c953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:27.746806 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31990a3d-cfe7-473e-bf64-c52fd954c953 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:34:27.822501 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:27.868537 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115e0969-8bc9-4afa-80ef-76e9e20d4d8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf14b64bf04b2ba74d2fe1fd41ff71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:34:27.869202 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115e0969-8bc9-4afa-80ef-76e9e20d4d8b service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:27.869477 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115e0969-8bc9-4afa-80ef-76e9e20d4d8b service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:27.869880 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115e0969-8bc9-4afa-80ef-76e9e20d4d8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:27.871591 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-115e0969-8bc9-4afa-80ef-76e9e20d4d8b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:27.974573 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115e0969-8bc9-4afa-80ef-76e9e20d4d8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:34:27.976329 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115e0969-8bc9-4afa-80ef-76e9e20d4d8b service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:34:27.996141 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 677/1360] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:34:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:34:30.367362 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb882a6-a3e0-42ba-8527-26886efa3c58 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:30.367804 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb882a6-a3e0-42ba-8527-26886efa3c58 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:30.368338 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb882a6-a3e0-42ba-8527-26886efa3c58 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:30.374041 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 684/1361] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:34:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:34:30.374814 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d3ca27-7c58-4ee3-a08f-34ca51ee55fd None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:30.375299 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d3ca27-7c58-4ee3-a08f-34ca51ee55fd None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:30.375299 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d3ca27-7c58-4ee3-a08f-34ca51ee55fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:30.452516 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-30d3ca27-7c58-4ee3-a08f-34ca51ee55fd None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:34:30.454490 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-30d3ca27-7c58-4ee3-a08f-34ca51ee55fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:30.496793 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:30.561692 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 678/1362] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:34:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:34:30.853067 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb882a6-a3e0-42ba-8527-26886efa3c58 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:34:30.854675 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4fb882a6-a3e0-42ba-8527-26886efa3c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:30.926998 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:31.032600 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb882a6-a3e0-42ba-8527-26886efa3c58 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:34:31.172360 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6997099c-039b-49f4-a97f-22ae5b0bf650 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf14b64bf04b2ba74d2fe1fd41ff71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:34:31.173027 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6997099c-039b-49f4-a97f-22ae5b0bf650 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:31.173696 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6997099c-039b-49f4-a97f-22ae5b0bf650 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:31.174106 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6997099c-039b-49f4-a97f-22ae5b0bf650 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:31.175757 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6997099c-039b-49f4-a97f-22ae5b0bf650 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:31.198849 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:31.268825 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6997099c-039b-49f4-a97f-22ae5b0bf650 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:34:31.270396 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6997099c-039b-49f4-a97f-22ae5b0bf650 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:34:31.289123 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 685/1363] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:34:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:34:33.799806 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18add03d-f8f2-46e5-b0a0-982ea757293d None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:33.800808 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18add03d-f8f2-46e5-b0a0-982ea757293d None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:33.801299 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18add03d-f8f2-46e5-b0a0-982ea757293d None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:33.802917 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 679/1364] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:34:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:34:33.808536 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72804d12-0b0a-40e6-b0c6-aa195bdd7db0 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:33.809345 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72804d12-0b0a-40e6-b0c6-aa195bdd7db0 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:33.809872 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72804d12-0b0a-40e6-b0c6-aa195bdd7db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:33.879417 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-72804d12-0b0a-40e6-b0c6-aa195bdd7db0 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:34:33.880993 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-72804d12-0b0a-40e6-b0c6-aa195bdd7db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:33.980727 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:33.990169 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 686/1365] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:34:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:34:34.273533 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb790cbb-d149-4a61-b8bb-a20005c7d3e5 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:34.273533 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb790cbb-d149-4a61-b8bb-a20005c7d3e5 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:34.273533 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb790cbb-d149-4a61-b8bb-a20005c7d3e5 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:34.274032 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 680/1366] 10.222.0.52 () {54 vars in 961 bytes} [Wed Sep 28 15:34:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:34:34.280068 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3556bbe-85cc-47fb-b165-81ceb21a3894 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:34.280582 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3556bbe-85cc-47fb-b165-81ceb21a3894 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:34.281080 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3556bbe-85cc-47fb-b165-81ceb21a3894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:34.369159 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-e3556bbe-85cc-47fb-b165-81ceb21a3894 None None] MFA Rules not processed for user `8daf14b64bf04b2ba74d2fe1fd41ff71`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:34:34.370608 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e3556bbe-85cc-47fb-b165-81ceb21a3894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:34.493250 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 687/1367] 10.222.0.52 () {58 vars in 1058 bytes} [Wed Sep 28 15:34:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:34:34.501622 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb790cbb-d149-4a61-b8bb-a20005c7d3e5 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:34:34.503388 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-cb790cbb-d149-4a61-b8bb-a20005c7d3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:34.658937 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb790cbb-d149-4a61-b8bb-a20005c7d3e5 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:34:34.682934 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:34.786970 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75873e1f-3005-46ee-ad00-231930419dd8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf14b64bf04b2ba74d2fe1fd41ff71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:34:34.787710 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75873e1f-3005-46ee-ad00-231930419dd8 service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:34.787990 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75873e1f-3005-46ee-ad00-231930419dd8 service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:34.788261 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75873e1f-3005-46ee-ad00-231930419dd8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:34.789878 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-75873e1f-3005-46ee-ad00-231930419dd8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:34.888360 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75873e1f-3005-46ee-ad00-231930419dd8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:34:34.889990 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75873e1f-3005-46ee-ad00-231930419dd8 service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:34:34.910489 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 681/1368] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:34:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:34:35.070739 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82728946-4547-4a3f-8a46-0c875d542713 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:35.071279 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82728946-4547-4a3f-8a46-0c875d542713 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:35.071398 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82728946-4547-4a3f-8a46-0c875d542713 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:35.072924 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 688/1369] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:34:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:34:35.079030 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a6f8d-9391-4fe5-9f71-d6b893dfc141 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:35.079361 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a6f8d-9391-4fe5-9f71-d6b893dfc141 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:35.079575 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a6f8d-9391-4fe5-9f71-d6b893dfc141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:35.117021 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:35.160110 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-919a6f8d-9391-4fe5-9f71-d6b893dfc141 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:34:35.161562 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-919a6f8d-9391-4fe5-9f71-d6b893dfc141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:35.282423 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 682/1370] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:34:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:34:35.615876 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63539d5d-ff9d-407e-9099-d4d1113736c5 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:35.615876 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63539d5d-ff9d-407e-9099-d4d1113736c5 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:35.615876 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63539d5d-ff9d-407e-9099-d4d1113736c5 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:35.617346 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 689/1371] 10.222.0.52 () {54 vars in 963 bytes} [Wed Sep 28 15:34:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:34:35.621529 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945a66a-7637-4723-906f-3c3a888b83b5 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:35.621682 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945a66a-7637-4723-906f-3c3a888b83b5 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:35.621948 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945a66a-7637-4723-906f-3c3a888b83b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:35.702644 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-5945a66a-7637-4723-906f-3c3a888b83b5 None None] MFA Rules not processed for user `01833709177e4e68a2fa52fb9963d3bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:34:35.703876 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-5945a66a-7637-4723-906f-3c3a888b83b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:35.753347 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:35.784823 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 683/1372] 10.222.0.52 () {58 vars in 1060 bytes} [Wed Sep 28 15:34:35 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:34:35.792486 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63539d5d-ff9d-407e-9099-d4d1113736c5 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:34:35.794175 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-63539d5d-ff9d-407e-9099-d4d1113736c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:35.850182 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:35.912783 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63539d5d-ff9d-407e-9099-d4d1113736c5 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:34:35.989726 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baa8806-bacc-4601-9cd3-0b8d18eb1a95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:34:35.990495 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa8806-bacc-4601-9cd3-0b8d18eb1a95 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:35.990871 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa8806-bacc-4601-9cd3-0b8d18eb1a95 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:35.991239 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa8806-bacc-4601-9cd3-0b8d18eb1a95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:35.992928 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7baa8806-bacc-4601-9cd3-0b8d18eb1a95 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:36.106406 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:36.127331 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baa8806-bacc-4601-9cd3-0b8d18eb1a95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:34:36.130621 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baa8806-bacc-4601-9cd3-0b8d18eb1a95 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:34:36.155749 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 690/1373] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:34:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:34:37.187258 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d381e8-ea76-47ab-97b0-e99dfada8911 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:37.188310 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d381e8-ea76-47ab-97b0-e99dfada8911 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:37.188400 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d381e8-ea76-47ab-97b0-e99dfada8911 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:37.189680 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 684/1374] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:34:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:34:37.194986 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53bd4a7-7b12-4963-8d24-d10a4adf9616 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:37.195376 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53bd4a7-7b12-4963-8d24-d10a4adf9616 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:37.195654 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53bd4a7-7b12-4963-8d24-d10a4adf9616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:37.261208 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-c53bd4a7-7b12-4963-8d24-d10a4adf9616 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:34:37.262435 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c53bd4a7-7b12-4963-8d24-d10a4adf9616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:37.360333 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 691/1375] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:34:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:34:37.648405 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d381e8-ea76-47ab-97b0-e99dfada8911 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:34:37.650067 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-06d381e8-ea76-47ab-97b0-e99dfada8911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:37.814024 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d381e8-ea76-47ab-97b0-e99dfada8911 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:34:37.984985 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:38.051815 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215df477-c23e-4496-941e-cfd1d1ef6169 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf14b64bf04b2ba74d2fe1fd41ff71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:34:38.052574 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215df477-c23e-4496-941e-cfd1d1ef6169 service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:38.052845 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215df477-c23e-4496-941e-cfd1d1ef6169 service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:38.053112 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215df477-c23e-4496-941e-cfd1d1ef6169 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:38.054915 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-215df477-c23e-4496-941e-cfd1d1ef6169 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:38.151720 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215df477-c23e-4496-941e-cfd1d1ef6169 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:34:38.153345 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215df477-c23e-4496-941e-cfd1d1ef6169 service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:34:38.172019 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 685/1376] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:34:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:34:40.387439 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc86e2-4a2c-40c2-83e7-57c55d5859e3 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:40.387963 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc86e2-4a2c-40c2-83e7-57c55d5859e3 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:40.387963 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc86e2-4a2c-40c2-83e7-57c55d5859e3 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:40.389912 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 692/1377] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:34:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:34:40.395341 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f8c7a7-6693-4faa-950a-48a7f3c65953 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:40.395798 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f8c7a7-6693-4faa-950a-48a7f3c65953 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:40.395893 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f8c7a7-6693-4faa-950a-48a7f3c65953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:40.464537 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-60f8c7a7-6693-4faa-950a-48a7f3c65953 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:34:40.465721 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-60f8c7a7-6693-4faa-950a-48a7f3c65953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:40.472990 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:40.573757 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 686/1378] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:34:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:34:40.865129 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dc86e2-4a2c-40c2-83e7-57c55d5859e3 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:34:40.866762 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-61dc86e2-4a2c-40c2-83e7-57c55d5859e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:40.932176 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:41.025136 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dc86e2-4a2c-40c2-83e7-57c55d5859e3 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:34:41.161569 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0fd1c1-699c-498a-890c-79986e2b8fad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf14b64bf04b2ba74d2fe1fd41ff71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:34:41.162111 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0fd1c1-699c-498a-890c-79986e2b8fad service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:41.162384 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0fd1c1-699c-498a-890c-79986e2b8fad service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:41.162656 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0fd1c1-699c-498a-890c-79986e2b8fad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:41.164226 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-da0fd1c1-699c-498a-890c-79986e2b8fad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:41.183773 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:41.259998 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0fd1c1-699c-498a-890c-79986e2b8fad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:34:41.261579 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0fd1c1-699c-498a-890c-79986e2b8fad service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:34:41.283981 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 693/1379] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:34:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:34:43.503273 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f08488-5863-4605-88b4-c69c57832c6f None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:43.503722 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f08488-5863-4605-88b4-c69c57832c6f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:43.503878 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f08488-5863-4605-88b4-c69c57832c6f None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:43.505317 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 687/1380] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:34:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:34:43.510703 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427a49b-2d0d-42c8-b9fe-7737d9903da3 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:43.511090 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427a49b-2d0d-42c8-b9fe-7737d9903da3 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:43.511268 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427a49b-2d0d-42c8-b9fe-7737d9903da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:43.585473 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-f427a49b-2d0d-42c8-b9fe-7737d9903da3 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:34:43.587310 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f427a49b-2d0d-42c8-b9fe-7737d9903da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:43.725787 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:43.731990 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 694/1381] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:34:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:34:44.027501 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f08488-5863-4605-88b4-c69c57832c6f None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:34:44.029000 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-23f08488-5863-4605-88b4-c69c57832c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:44.175467 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:44.189664 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f08488-5863-4605-88b4-c69c57832c6f None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:34:44.320741 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a21bac0-a863-4c5f-b3eb-4093adcb5685 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf14b64bf04b2ba74d2fe1fd41ff71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:34:44.321411 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21bac0-a863-4c5f-b3eb-4093adcb5685 service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:44.321721 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21bac0-a863-4c5f-b3eb-4093adcb5685 service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:44.322046 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21bac0-a863-4c5f-b3eb-4093adcb5685 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:44.323821 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-6a21bac0-a863-4c5f-b3eb-4093adcb5685 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:44.413964 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a21bac0-a863-4c5f-b3eb-4093adcb5685 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:34:44.415543 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a21bac0-a863-4c5f-b3eb-4093adcb5685 service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:34:44.425444 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:44.434390 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 688/1382] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:34:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:34:46.607829 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b595b24-d695-44fe-934a-bafac29e5991 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:46.608373 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b595b24-d695-44fe-934a-bafac29e5991 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:46.608373 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b595b24-d695-44fe-934a-bafac29e5991 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:46.609780 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 695/1383] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:34:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:34:46.615244 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e54c4-30fd-4bf0-9036-46043a1fd703 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:46.615749 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e54c4-30fd-4bf0-9036-46043a1fd703 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:46.615933 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486e54c4-30fd-4bf0-9036-46043a1fd703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:46.681900 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-486e54c4-30fd-4bf0-9036-46043a1fd703 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:34:46.683214 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-486e54c4-30fd-4bf0-9036-46043a1fd703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:46.792706 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 689/1384] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:34:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:34:47.116407 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b595b24-d695-44fe-934a-bafac29e5991 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:34:47.118439 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7b595b24-d695-44fe-934a-bafac29e5991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:47.299054 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b595b24-d695-44fe-934a-bafac29e5991 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:34:47.378493 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:47.426919 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514cef73-61c0-496b-a5fe-66f4e81cdbe2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf14b64bf04b2ba74d2fe1fd41ff71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:34:47.427598 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514cef73-61c0-496b-a5fe-66f4e81cdbe2 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:47.428045 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514cef73-61c0-496b-a5fe-66f4e81cdbe2 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:47.428386 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514cef73-61c0-496b-a5fe-66f4e81cdbe2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:47.430223 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-514cef73-61c0-496b-a5fe-66f4e81cdbe2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:47.536300 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514cef73-61c0-496b-a5fe-66f4e81cdbe2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:34:47.537976 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514cef73-61c0-496b-a5fe-66f4e81cdbe2 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:34:47.556137 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 696/1385] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:34:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:34:49.722074 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec797c-c71f-4a0f-917f-099edd1cd890 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:49.722624 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec797c-c71f-4a0f-917f-099edd1cd890 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:49.722624 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec797c-c71f-4a0f-917f-099edd1cd890 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:49.724168 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 690/1386] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:34:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:34:49.729154 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fc1dd1-835d-4d50-a478-f3459b5d7050 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:49.729720 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fc1dd1-835d-4d50-a478-f3459b5d7050 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:49.730036 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fc1dd1-835d-4d50-a478-f3459b5d7050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:49.795515 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-13fc1dd1-835d-4d50-a478-f3459b5d7050 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:34:49.796692 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-13fc1dd1-835d-4d50-a478-f3459b5d7050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:49.829973 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:49.895876 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 697/1387] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:34:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:34:50.186727 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ec797c-c71f-4a0f-917f-099edd1cd890 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:34:50.188840 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c1ec797c-c71f-4a0f-917f-099edd1cd890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:50.264903 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:50.338095 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ec797c-c71f-4a0f-917f-099edd1cd890 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:34:50.448177 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb7c375-82cf-4055-8426-da41c1c88e58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf14b64bf04b2ba74d2fe1fd41ff71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:34:50.448725 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb7c375-82cf-4055-8426-da41c1c88e58 service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:50.449022 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb7c375-82cf-4055-8426-da41c1c88e58 service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:50.449295 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb7c375-82cf-4055-8426-da41c1c88e58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:50.450902 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-aeb7c375-82cf-4055-8426-da41c1c88e58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:50.489450 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:50.542424 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb7c375-82cf-4055-8426-da41c1c88e58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:34:50.544002 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb7c375-82cf-4055-8426-da41c1c88e58 service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:34:50.561538 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 691/1388] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:34:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:34:51.432237 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89dd3e-62ba-4751-a428-9994d4704dca None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:51.432813 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89dd3e-62ba-4751-a428-9994d4704dca None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:51.432813 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89dd3e-62ba-4751-a428-9994d4704dca None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:51.434321 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 698/1389] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:34:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:34:51.439568 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb758e-7e15-4959-b861-64d544df458b None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:51.439996 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb758e-7e15-4959-b861-64d544df458b None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:51.440109 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb758e-7e15-4959-b861-64d544df458b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:51.511250 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-37cb758e-7e15-4959-b861-64d544df458b None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:34:51.512447 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-37cb758e-7e15-4959-b861-64d544df458b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:51.608087 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 692/1390] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:34:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:34:51.914363 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f02491-45bf-4559-8609-781a0a547075 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:51.914644 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f02491-45bf-4559-8609-781a0a547075 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:51.914934 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f02491-45bf-4559-8609-781a0a547075 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:51.916310 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 699/1391] 10.222.0.52 () {54 vars in 962 bytes} [Wed Sep 28 15:34:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:34:51.924634 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21169e-cd1a-4b34-80c4-8f7fd91901a0 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:51.924918 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21169e-cd1a-4b34-80c4-8f7fd91901a0 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:51.925186 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21169e-cd1a-4b34-80c4-8f7fd91901a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:51.945239 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:52.005406 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-2f21169e-cd1a-4b34-80c4-8f7fd91901a0 None None] MFA Rules not processed for user `c6a46bbd249c4434ad7d9442754f098a`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:34:52.006635 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-2f21169e-cd1a-4b34-80c4-8f7fd91901a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:52.080962 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 693/1392] 10.222.0.52 () {58 vars in 1059 bytes} [Wed Sep 28 15:34:51 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:34:52.090166 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f02491-45bf-4559-8609-781a0a547075 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:34:52.091697 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-07f02491-45bf-4559-8609-781a0a547075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:52.198153 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f02491-45bf-4559-8609-781a0a547075 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:34:52.257873 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:52.275326 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf1dfe-4284-4639-bffb-f425a833d2c0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6a46bbd249c4434ad7d9442754f098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:34:52.276191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf1dfe-4284-4639-bffb-f425a833d2c0 service glance] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:52.276651 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf1dfe-4284-4639-bffb-f425a833d2c0 service glance] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:52.277086 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf1dfe-4284-4639-bffb-f425a833d2c0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:52.278916 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e4bf1dfe-4284-4639-bffb-f425a833d2c0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:52.397980 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bf1dfe-4284-4639-bffb-f425a833d2c0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:34:52.399435 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bf1dfe-4284-4639-bffb-f425a833d2c0 service glance] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:34:52.417123 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 700/1393] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:34:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:34:52.742723 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2751d-d614-411b-84ce-04bedfd2b0af None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:52.743305 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2751d-d614-411b-84ce-04bedfd2b0af None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:52.743305 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2751d-d614-411b-84ce-04bedfd2b0af None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:52.744588 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 694/1394] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:34:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:34:52.751353 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f25949c-dbcc-4ae8-8669-816191fd89de None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:52.751761 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f25949c-dbcc-4ae8-8669-816191fd89de None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:52.751908 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f25949c-dbcc-4ae8-8669-816191fd89de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:52.825740 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-5f25949c-dbcc-4ae8-8669-816191fd89de None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:34:52.826968 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5f25949c-dbcc-4ae8-8669-816191fd89de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:52.874303 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:52.933898 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 701/1395] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:34:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:34:53.160192 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc262d-e843-4076-9eb3-2a23b0694af9 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:53.160475 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc262d-e843-4076-9eb3-2a23b0694af9 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:53.160744 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc262d-e843-4076-9eb3-2a23b0694af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:53.226414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-83dc262d-e843-4076-9eb3-2a23b0694af9 None None] MFA Rules not processed for user `c81e5fb007614cf2992f5e02e85d6307`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:34:53.227599 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-83dc262d-e843-4076-9eb3-2a23b0694af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:53.234960 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f25949c-dbcc-4ae8-8669-816191fd89de None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:34:53.236138 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5f25949c-dbcc-4ae8-8669-816191fd89de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:53.285989 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:53.335660 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 695/1396] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Sep 28 15:34:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:34:53.377568 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f25949c-dbcc-4ae8-8669-816191fd89de None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:34:53.380808 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d10cbf-2739-4c0e-a30c-aa1e995a9e88 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:53.381098 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d10cbf-2739-4c0e-a30c-aa1e995a9e88 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:53.381381 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d10cbf-2739-4c0e-a30c-aa1e995a9e88 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:53.382801 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 696/1397] 10.222.0.52 () {54 vars in 943 bytes} [Wed Sep 28 15:34:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:34:53.389174 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b0cd19-a797-43de-9cc9-992dde11f96f None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:53.389476 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b0cd19-a797-43de-9cc9-992dde11f96f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:53.389752 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b0cd19-a797-43de-9cc9-992dde11f96f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:53.433627 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:53.468367 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-66b0cd19-a797-43de-9cc9-992dde11f96f None None] MFA Rules not processed for user `328051fe7607464f8f5dc0dcc01a817a`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:34:53.470531 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-66b0cd19-a797-43de-9cc9-992dde11f96f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:53.492811 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f016e5-5b73-4c43-acbf-acc831581679 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf14b64bf04b2ba74d2fe1fd41ff71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:34:53.493415 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f016e5-5b73-4c43-acbf-acc831581679 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:53.493686 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f016e5-5b73-4c43-acbf-acc831581679 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:53.493986 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f016e5-5b73-4c43-acbf-acc831581679 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:53.495831 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c4f016e5-5b73-4c43-acbf-acc831581679 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:53.580425 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 697/1398] 10.222.0.52 () {58 vars in 1040 bytes} [Wed Sep 28 15:34:53 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:34:53.587145 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f016e5-5b73-4c43-acbf-acc831581679 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:34:53.588822 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f016e5-5b73-4c43-acbf-acc831581679 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:34:53.589659 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b0cd19-a797-43de-9cc9-992dde11f96f None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:34:53.598903 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b0cd19-a797-43de-9cc9-992dde11f96f None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:34:53.607007 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 702/1399] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:34:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:34:53.628920 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:53.715392 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2f0cd2-a711-440a-a230-bcbe8fcd4e9a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '328051fe7607464f8f5dc0dcc01a817a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:34:53.716007 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f0cd2-a711-440a-a230-bcbe8fcd4e9a service swift] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:53.716281 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f0cd2-a711-440a-a230-bcbe8fcd4e9a service swift] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:53.716547 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2f0cd2-a711-440a-a230-bcbe8fcd4e9a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:53.718397 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-9a2f0cd2-a711-440a-a230-bcbe8fcd4e9a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:53.812342 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f0cd2-a711-440a-a230-bcbe8fcd4e9a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:34:53.814662 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2f0cd2-a711-440a-a230-bcbe8fcd4e9a service swift] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:34:53.823660 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 698/1400] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 15:34:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 28 15:34:55.812567 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f106f-741e-4470-a6e6-f5fab97a2827 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:55.814229 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f106f-741e-4470-a6e6-f5fab97a2827 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:55.816551 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f106f-741e-4470-a6e6-f5fab97a2827 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:55.818138 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 703/1401] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:34:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:34:55.823636 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10ea2b-5120-4d24-9999-fa6ca582c6ba None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:55.824120 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10ea2b-5120-4d24-9999-fa6ca582c6ba None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:55.824226 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10ea2b-5120-4d24-9999-fa6ca582c6ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:55.897312 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-6f10ea2b-5120-4d24-9999-fa6ca582c6ba None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:34:55.898548 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-6f10ea2b-5120-4d24-9999-fa6ca582c6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:55.905267 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:56.012970 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 699/1402] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:34:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:34:56.303326 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f106f-741e-4470-a6e6-f5fab97a2827 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:34:56.305131 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-fd3f106f-741e-4470-a6e6-f5fab97a2827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:56.378778 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:56.484302 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f106f-741e-4470-a6e6-f5fab97a2827 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:34:56.621285 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c68a372-f296-48b5-9f6a-2b92c55749ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf14b64bf04b2ba74d2fe1fd41ff71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:34:56.622173 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68a372-f296-48b5-9f6a-2b92c55749ca service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:56.622760 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68a372-f296-48b5-9f6a-2b92c55749ca service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:56.623392 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68a372-f296-48b5-9f6a-2b92c55749ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:56.625286 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1c68a372-f296-48b5-9f6a-2b92c55749ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:56.662227 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:56.741584 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c68a372-f296-48b5-9f6a-2b92c55749ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:34:56.743337 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c68a372-f296-48b5-9f6a-2b92c55749ca service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:34:56.765555 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 704/1403] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:34:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:34:59.081481 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b5496-d3a4-47f6-ae27-c1574154875c None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:59.082083 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b5496-d3a4-47f6-ae27-c1574154875c None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:59.082083 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069b5496-d3a4-47f6-ae27-c1574154875c None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:59.083394 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 700/1404] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:34:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:34:59.088605 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c8fbe-8569-4427-ac5e-1d7c9e5c8275 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:59.089013 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c8fbe-8569-4427-ac5e-1d7c9e5c8275 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:59.089166 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c8fbe-8569-4427-ac5e-1d7c9e5c8275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:59.160928 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-652c8fbe-8569-4427-ac5e-1d7c9e5c8275 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:34:59.162187 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-652c8fbe-8569-4427-ac5e-1d7c9e5c8275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:59.267819 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 705/1405] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:34:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:34:59.557288 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069b5496-d3a4-47f6-ae27-c1574154875c None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:34:59.558845 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-069b5496-d3a4-47f6-ae27-c1574154875c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:59.732414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:59.748012 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069b5496-d3a4-47f6-ae27-c1574154875c None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:34:59.872965 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253bf09f-1f08-47f7-9393-5a41078e0a66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf14b64bf04b2ba74d2fe1fd41ff71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:34:59.873692 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253bf09f-1f08-47f7-9393-5a41078e0a66 service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:34:59.874010 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253bf09f-1f08-47f7-9393-5a41078e0a66 service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:34:59.874277 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253bf09f-1f08-47f7-9393-5a41078e0a66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:34:59.876055 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-253bf09f-1f08-47f7-9393-5a41078e0a66 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:34:59.973663 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253bf09f-1f08-47f7-9393-5a41078e0a66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:34:59.975566 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253bf09f-1f08-47f7-9393-5a41078e0a66 service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:34:59.986604 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:34:59.996844 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 701/1406] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:34:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:35:02.222571 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a551ae-bb97-4184-8c37-3aad4ddbb537 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:02.223154 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a551ae-bb97-4184-8c37-3aad4ddbb537 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:02.223154 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a551ae-bb97-4184-8c37-3aad4ddbb537 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:02.224456 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 706/1407] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:35:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:35:02.229823 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c8555-156a-4ef4-aa59-5925a9933490 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:02.230274 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c8555-156a-4ef4-aa59-5925a9933490 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:02.230413 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c8555-156a-4ef4-aa59-5925a9933490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:02.307990 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-dd7c8555-156a-4ef4-aa59-5925a9933490 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:35:02.309195 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-dd7c8555-156a-4ef4-aa59-5925a9933490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:02.425272 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 702/1408] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:35:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:35:02.716360 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a551ae-bb97-4184-8c37-3aad4ddbb537 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:35:02.718253 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-26a551ae-bb97-4184-8c37-3aad4ddbb537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:02.722256 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:02.910109 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a551ae-bb97-4184-8c37-3aad4ddbb537 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:35:02.992204 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:03.033523 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf168f3-c064-4d19-bae6-31d99f63526e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf14b64bf04b2ba74d2fe1fd41ff71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:35:03.034165 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf168f3-c064-4d19-bae6-31d99f63526e service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:03.034446 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf168f3-c064-4d19-bae6-31d99f63526e service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:03.034719 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf168f3-c064-4d19-bae6-31d99f63526e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:03.036305 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1bf168f3-c064-4d19-bae6-31d99f63526e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:03.150320 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf168f3-c064-4d19-bae6-31d99f63526e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:35:03.152805 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf168f3-c064-4d19-bae6-31d99f63526e service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:35:03.174033 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 707/1409] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:35:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:35:12.423729 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=67 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:13.170907 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=61 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:16.743879 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc2517-1d0c-4640-b11b-b8a133c69d75 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:16.744455 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc2517-1d0c-4640-b11b-b8a133c69d75 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:16.744455 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc2517-1d0c-4640-b11b-b8a133c69d75 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:16.745729 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 703/1410] 10.222.0.52 () {54 vars in 918 bytes} [Wed Sep 28 15:35:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:35:16.751672 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07d67b-0e34-460c-b0c3-1870952d8cb8 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:16.752079 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07d67b-0e34-460c-b0c3-1870952d8cb8 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:16.752262 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc07d67b-0e34-460c-b0c3-1870952d8cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:16.826299 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-fc07d67b-0e34-460c-b0c3-1870952d8cb8 None None] MFA Rules not processed for user `de934d0105bd4d01ba6c06a4a0d36704`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:35:16.827478 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-fc07d67b-0e34-460c-b0c3-1870952d8cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:16.932674 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 708/1411] 10.222.0.52 () {58 vars in 1015 bytes} [Wed Sep 28 15:35:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:35:16.947413 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b9bf5a-3e50-4a4f-8841-2d808fc7e907 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:16.947689 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b9bf5a-3e50-4a4f-8841-2d808fc7e907 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:16.947958 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b9bf5a-3e50-4a4f-8841-2d808fc7e907 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:16.949252 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 704/1412] 10.222.0.52 () {54 vars in 918 bytes} [Wed Sep 28 15:35:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:35:16.953982 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c042071-5303-4322-92b1-a19f75de9ce1 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:16.954299 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c042071-5303-4322-92b1-a19f75de9ce1 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:16.954589 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c042071-5303-4322-92b1-a19f75de9ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:17.026475 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:17.029453 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-6c042071-5303-4322-92b1-a19f75de9ce1 None None] MFA Rules not processed for user `8daf14b64bf04b2ba74d2fe1fd41ff71`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:35:17.031226 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6c042071-5303-4322-92b1-a19f75de9ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:17.143998 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 709/1413] 10.222.0.52 () {58 vars in 1015 bytes} [Wed Sep 28 15:35:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:35:19.730600 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db31fd-511f-4538-96ed-76065bcc87e4 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:19.731164 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db31fd-511f-4538-96ed-76065bcc87e4 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:19.731164 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db31fd-511f-4538-96ed-76065bcc87e4 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:19.732449 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 705/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 15:35:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:35:19.737943 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7961e-984e-41ff-b093-0106d92e0670 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:19.738384 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7961e-984e-41ff-b093-0106d92e0670 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:19.738490 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7961e-984e-41ff-b093-0106d92e0670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:19.805166 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-fce7961e-984e-41ff-b093-0106d92e0670 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:35:19.806397 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-fce7961e-984e-41ff-b093-0106d92e0670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:19.864515 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:19.906355 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 710/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 15:35:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:35:20.195182 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35db31fd-511f-4538-96ed-76065bcc87e4 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:35:20.196756 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-35db31fd-511f-4538-96ed-76065bcc87e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:20.349101 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35db31fd-511f-4538-96ed-76065bcc87e4 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:35:20.420448 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:20.460193 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb52e42-986d-4594-ae7b-207e64c1ac75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf14b64bf04b2ba74d2fe1fd41ff71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:35:20.460727 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb52e42-986d-4594-ae7b-207e64c1ac75 service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:20.460999 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb52e42-986d-4594-ae7b-207e64c1ac75 service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:20.461265 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb52e42-986d-4594-ae7b-207e64c1ac75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:20.462871 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-bdb52e42-986d-4594-ae7b-207e64c1ac75 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:20.550673 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb52e42-986d-4594-ae7b-207e64c1ac75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:35:20.552236 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb52e42-986d-4594-ae7b-207e64c1ac75 service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:35:20.569754 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 706/1416] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:35:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:35:21.722374 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31c753-bf5e-46f0-a959-27bce2f07db8 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:21.722925 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31c753-bf5e-46f0-a959-27bce2f07db8 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:21.722925 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31c753-bf5e-46f0-a959-27bce2f07db8 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:21.724257 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 711/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 15:35:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:35:21.730968 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df0be2a-cee1-44d9-aeef-d71b22e662b3 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:21.731828 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df0be2a-cee1-44d9-aeef-d71b22e662b3 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:21.732110 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df0be2a-cee1-44d9-aeef-d71b22e662b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:21.807311 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-0df0be2a-cee1-44d9-aeef-d71b22e662b3 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:35:21.808501 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-0df0be2a-cee1-44d9-aeef-d71b22e662b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:21.835230 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:21.903645 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 707/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 15:35:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:35:22.073193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b31c753-bf5e-46f0-a959-27bce2f07db8 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:35:22.074726 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6b31c753-bf5e-46f0-a959-27bce2f07db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:22.218797 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b31c753-bf5e-46f0-a959-27bce2f07db8 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:35:22.259586 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:22.332948 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb519061-478c-4e31-aa26-061de75b6f0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf14b64bf04b2ba74d2fe1fd41ff71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:35:22.333485 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb519061-478c-4e31-aa26-061de75b6f0d service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:22.333762 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb519061-478c-4e31-aa26-061de75b6f0d service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:22.334065 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb519061-478c-4e31-aa26-061de75b6f0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:22.335654 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-eb519061-478c-4e31-aa26-061de75b6f0d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:22.426296 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb519061-478c-4e31-aa26-061de75b6f0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:35:22.427875 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb519061-478c-4e31-aa26-061de75b6f0d service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:35:22.445911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 712/1419] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:35:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:35:23.578290 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0261977b-6b16-4a9e-a3f7-5be119974296 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:23.578821 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0261977b-6b16-4a9e-a3f7-5be119974296 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:23.578821 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0261977b-6b16-4a9e-a3f7-5be119974296 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:23.580684 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 708/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 15:35:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:35:23.586773 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8168c57-99ef-4c44-8e3f-6c2091b6b6bb None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:23.587194 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8168c57-99ef-4c44-8e3f-6c2091b6b6bb None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:23.587336 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8168c57-99ef-4c44-8e3f-6c2091b6b6bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:23.640619 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:23.663709 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-a8168c57-99ef-4c44-8e3f-6c2091b6b6bb None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:35:23.664898 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a8168c57-99ef-4c44-8e3f-6c2091b6b6bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:23.758373 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 713/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 15:35:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:35:23.913571 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0261977b-6b16-4a9e-a3f7-5be119974296 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:35:23.915026 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-0261977b-6b16-4a9e-a3f7-5be119974296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:24.059014 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0261977b-6b16-4a9e-a3f7-5be119974296 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:35:24.072065 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:24.171333 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd948e5-950d-4555-a881-b0551c887836 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf14b64bf04b2ba74d2fe1fd41ff71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:35:24.171858 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd948e5-950d-4555-a881-b0551c887836 service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:24.172132 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd948e5-950d-4555-a881-b0551c887836 service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:24.172399 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd948e5-950d-4555-a881-b0551c887836 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:24.174018 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3fd948e5-950d-4555-a881-b0551c887836 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:24.262233 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd948e5-950d-4555-a881-b0551c887836 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:35:24.263811 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd948e5-950d-4555-a881-b0551c887836 service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:35:24.285258 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 709/1422] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:35:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:35:25.392887 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a697254-83c7-4361-b4cd-544b4706fb77 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:25.393369 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a697254-83c7-4361-b4cd-544b4706fb77 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:25.393473 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a697254-83c7-4361-b4cd-544b4706fb77 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:25.394798 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 714/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 15:35:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:35:25.401528 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9216cdd-7350-40ca-acb6-0b2c3a1ac51b None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:25.401974 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9216cdd-7350-40ca-acb6-0b2c3a1ac51b None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:25.402094 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9216cdd-7350-40ca-acb6-0b2c3a1ac51b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:25.421298 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:25.477271 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-e9216cdd-7350-40ca-acb6-0b2c3a1ac51b None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:35:25.478597 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e9216cdd-7350-40ca-acb6-0b2c3a1ac51b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:25.573273 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 710/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 15:35:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:35:25.723737 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a697254-83c7-4361-b4cd-544b4706fb77 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:35:25.725278 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1a697254-83c7-4361-b4cd-544b4706fb77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:25.835935 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:25.871603 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a697254-83c7-4361-b4cd-544b4706fb77 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:35:25.981263 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a78e7f2-2a78-44e5-9860-59aa1542ecf9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf14b64bf04b2ba74d2fe1fd41ff71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:35:25.981883 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78e7f2-2a78-44e5-9860-59aa1542ecf9 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:25.982164 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78e7f2-2a78-44e5-9860-59aa1542ecf9 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:25.982437 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78e7f2-2a78-44e5-9860-59aa1542ecf9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:25.984013 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6a78e7f2-2a78-44e5-9860-59aa1542ecf9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:26.059204 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:26.073038 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a78e7f2-2a78-44e5-9860-59aa1542ecf9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:35:26.074649 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a78e7f2-2a78-44e5-9860-59aa1542ecf9 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:35:26.092544 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 715/1425] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:35:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:35:27.209828 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0687fe34-7235-46a3-8ac0-92d14e6a6879 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:27.210431 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0687fe34-7235-46a3-8ac0-92d14e6a6879 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:27.210431 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0687fe34-7235-46a3-8ac0-92d14e6a6879 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:27.211729 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 711/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 15:35:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:35:27.217835 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ddefa4-b33c-4158-a581-7fd2b5314db1 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:27.218342 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ddefa4-b33c-4158-a581-7fd2b5314db1 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:27.218447 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ddefa4-b33c-4158-a581-7fd2b5314db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:27.294260 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-c8ddefa4-b33c-4158-a581-7fd2b5314db1 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:35:27.295864 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c8ddefa4-b33c-4158-a581-7fd2b5314db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:27.401803 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 716/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 15:35:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:35:27.553429 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0687fe34-7235-46a3-8ac0-92d14e6a6879 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:35:27.554960 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-0687fe34-7235-46a3-8ac0-92d14e6a6879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:27.644181 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:27.706075 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0687fe34-7235-46a3-8ac0-92d14e6a6879 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:35:27.816778 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0204409b-50d5-4009-9a5a-fc19242a2a00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf14b64bf04b2ba74d2fe1fd41ff71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:35:27.817289 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0204409b-50d5-4009-9a5a-fc19242a2a00 service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:27.817558 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0204409b-50d5-4009-9a5a-fc19242a2a00 service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:27.817821 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0204409b-50d5-4009-9a5a-fc19242a2a00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:27.819402 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-0204409b-50d5-4009-9a5a-fc19242a2a00 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:27.867887 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:27.908611 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0204409b-50d5-4009-9a5a-fc19242a2a00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:35:27.910218 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0204409b-50d5-4009-9a5a-fc19242a2a00 service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:35:27.928220 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 712/1428] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:35:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:35:28.989766 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5723696-cec5-4163-b36d-d8f7894405f1 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:28.990670 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5723696-cec5-4163-b36d-d8f7894405f1 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:28.990670 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5723696-cec5-4163-b36d-d8f7894405f1 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:28.991667 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 717/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 15:35:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:35:28.998500 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd436c-e588-4007-836d-7bc092bd54ef None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:28.998966 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd436c-e588-4007-836d-7bc092bd54ef None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:28.999062 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfd436c-e588-4007-836d-7bc092bd54ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:29.074801 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-fbfd436c-e588-4007-836d-7bc092bd54ef None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:35:29.076001 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-fbfd436c-e588-4007-836d-7bc092bd54ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:29.171770 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 713/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 15:35:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:35:29.324703 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5723696-cec5-4163-b36d-d8f7894405f1 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:35:29.326245 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e5723696-cec5-4163-b36d-d8f7894405f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:29.375213 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:29.472345 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5723696-cec5-4163-b36d-d8f7894405f1 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:35:29.583725 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7b5ddf-5ac1-4ecc-9bb8-b9a77498faf1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf14b64bf04b2ba74d2fe1fd41ff71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:35:29.584328 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b5ddf-5ac1-4ecc-9bb8-b9a77498faf1 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:29.584615 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b5ddf-5ac1-4ecc-9bb8-b9a77498faf1 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:29.584885 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b5ddf-5ac1-4ecc-9bb8-b9a77498faf1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:29.586506 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-dc7b5ddf-5ac1-4ecc-9bb8-b9a77498faf1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:29.593220 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:29.679318 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7b5ddf-5ac1-4ecc-9bb8-b9a77498faf1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:35:29.680917 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7b5ddf-5ac1-4ecc-9bb8-b9a77498faf1 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:35:29.700376 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 718/1431] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:35:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:35:30.754657 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f9da06-de62-40f3-a3a4-505c5db79bd1 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:30.755246 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f9da06-de62-40f3-a3a4-505c5db79bd1 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:30.755246 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f9da06-de62-40f3-a3a4-505c5db79bd1 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:30.756524 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 714/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 15:35:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:35:30.762883 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d55a19-ff22-4dec-9ba6-fc66c1d3d262 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:30.763361 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d55a19-ff22-4dec-9ba6-fc66c1d3d262 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:30.763465 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d55a19-ff22-4dec-9ba6-fc66c1d3d262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:30.840635 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-85d55a19-ff22-4dec-9ba6-fc66c1d3d262 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:35:30.841836 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-85d55a19-ff22-4dec-9ba6-fc66c1d3d262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:30.910308 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:30.940015 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 719/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 15:35:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:35:31.088710 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f9da06-de62-40f3-a3a4-505c5db79bd1 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:35:31.090270 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-90f9da06-de62-40f3-a3a4-505c5db79bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:31.108808 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:31.233493 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f9da06-de62-40f3-a3a4-505c5db79bd1 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:35:31.321816 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:31.347895 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12170749-7d08-4fc5-a636-789311c7ff1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf14b64bf04b2ba74d2fe1fd41ff71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:35:31.348503 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12170749-7d08-4fc5-a636-789311c7ff1d service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:31.348777 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12170749-7d08-4fc5-a636-789311c7ff1d service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:31.349043 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12170749-7d08-4fc5-a636-789311c7ff1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:31.350698 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-12170749-7d08-4fc5-a636-789311c7ff1d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:31.438818 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12170749-7d08-4fc5-a636-789311c7ff1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:35:31.440383 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12170749-7d08-4fc5-a636-789311c7ff1d service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:35:31.458013 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 715/1434] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:35:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:35:32.505672 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed0d36-b000-4788-afd6-3a004b5e42ae None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:32.506247 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed0d36-b000-4788-afd6-3a004b5e42ae None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:32.506247 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed0d36-b000-4788-afd6-3a004b5e42ae None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:32.507619 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 720/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 15:35:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:35:32.513869 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22dcd19-94b0-4ef4-ae3b-ae8bb392c4e9 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:32.514329 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22dcd19-94b0-4ef4-ae3b-ae8bb392c4e9 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:32.514428 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22dcd19-94b0-4ef4-ae3b-ae8bb392c4e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:32.589405 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-c22dcd19-94b0-4ef4-ae3b-ae8bb392c4e9 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:35:32.590623 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c22dcd19-94b0-4ef4-ae3b-ae8bb392c4e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:32.632725 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:32.686303 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 716/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 15:35:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:35:32.834391 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffed0d36-b000-4788-afd6-3a004b5e42ae None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:35:32.835899 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ffed0d36-b000-4788-afd6-3a004b5e42ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:32.980576 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffed0d36-b000-4788-afd6-3a004b5e42ae None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:35:33.035673 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:33.098270 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f48468-84a0-4c76-8115-d4c319e2990d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf14b64bf04b2ba74d2fe1fd41ff71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:35:33.098874 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f48468-84a0-4c76-8115-d4c319e2990d service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:33.099151 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f48468-84a0-4c76-8115-d4c319e2990d service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:33.099473 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f48468-84a0-4c76-8115-d4c319e2990d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:33.101015 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-04f48468-84a0-4c76-8115-d4c319e2990d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:33.187672 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f48468-84a0-4c76-8115-d4c319e2990d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:35:33.189281 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f48468-84a0-4c76-8115-d4c319e2990d service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:35:33.209043 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 721/1437] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:35:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:35:34.257085 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6e25a-8e45-48ad-84be-039056f9c4b2 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:34.257688 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6e25a-8e45-48ad-84be-039056f9c4b2 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:34.257688 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6e25a-8e45-48ad-84be-039056f9c4b2 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:34.258995 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 717/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 15:35:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:35:34.265329 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245d62c4-a9e1-46bb-9492-c3e72955785f None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:34.265751 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245d62c4-a9e1-46bb-9492-c3e72955785f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:34.265889 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245d62c4-a9e1-46bb-9492-c3e72955785f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:34.340289 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-245d62c4-a9e1-46bb-9492-c3e72955785f None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:35:34.341480 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-245d62c4-a9e1-46bb-9492-c3e72955785f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:34.343486 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:34.439854 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 722/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 15:35:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:35:34.589317 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda6e25a-8e45-48ad-84be-039056f9c4b2 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:35:34.590917 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-bda6e25a-8e45-48ad-84be-039056f9c4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:34.735628 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda6e25a-8e45-48ad-84be-039056f9c4b2 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:35:34.761877 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:34.847634 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f24e31-c558-4af3-924c-8e434eebae80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf14b64bf04b2ba74d2fe1fd41ff71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:35:34.848155 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f24e31-c558-4af3-924c-8e434eebae80 service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:34.848428 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f24e31-c558-4af3-924c-8e434eebae80 service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:34.848696 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f24e31-c558-4af3-924c-8e434eebae80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:34.850316 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-47f24e31-c558-4af3-924c-8e434eebae80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:34.937644 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f24e31-c558-4af3-924c-8e434eebae80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:35:34.939726 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f24e31-c558-4af3-924c-8e434eebae80 service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:35:34.957782 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 718/1440] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:35:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:35:37.091207 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6642436-a6c5-4c23-a024-0081d8041409 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:37.091748 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6642436-a6c5-4c23-a024-0081d8041409 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:37.091748 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6642436-a6c5-4c23-a024-0081d8041409 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:37.093084 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 723/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 15:35:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:35:37.098836 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1702f6-bcde-4ec8-be81-5faa0f5e24a8 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:37.099243 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1702f6-bcde-4ec8-be81-5faa0f5e24a8 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:37.099404 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1702f6-bcde-4ec8-be81-5faa0f5e24a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:37.140090 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:37.165414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-0c1702f6-bcde-4ec8-be81-5faa0f5e24a8 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:35:37.166623 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-0c1702f6-bcde-4ec8-be81-5faa0f5e24a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:37.261503 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 719/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 15:35:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:35:37.551569 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6642436-a6c5-4c23-a024-0081d8041409 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:35:37.553100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f6642436-a6c5-4c23-a024-0081d8041409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:37.654400 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6642436-a6c5-4c23-a024-0081d8041409 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:35:37.685984 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:37.728721 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcddbd8-2899-4a3d-b448-19ccc01b3072 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:35:37.729321 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcddbd8-2899-4a3d-b448-19ccc01b3072 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:37.729600 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcddbd8-2899-4a3d-b448-19ccc01b3072 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:37.729893 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcddbd8-2899-4a3d-b448-19ccc01b3072 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:37.731831 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8bcddbd8-2899-4a3d-b448-19ccc01b3072 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:37.838123 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcddbd8-2899-4a3d-b448-19ccc01b3072 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:35:37.839672 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcddbd8-2899-4a3d-b448-19ccc01b3072 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:35:37.858048 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 724/1443] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:35:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:35:40.168876 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4cfedf-c61a-4876-b69b-7a340dea961c None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:40.169452 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4cfedf-c61a-4876-b69b-7a340dea961c None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:40.169452 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4cfedf-c61a-4876-b69b-7a340dea961c None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:40.170832 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 720/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 15:35:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:35:40.176708 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aceca1-e230-4ac7-97ec-e82be83918e7 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:40.177130 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aceca1-e230-4ac7-97ec-e82be83918e7 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:40.177291 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aceca1-e230-4ac7-97ec-e82be83918e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:40.241539 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-29aceca1-e230-4ac7-97ec-e82be83918e7 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:35:40.242761 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-29aceca1-e230-4ac7-97ec-e82be83918e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:40.255460 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:40.338678 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 725/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 15:35:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:35:40.621022 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4cfedf-c61a-4876-b69b-7a340dea961c None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:35:40.622584 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-8a4cfedf-c61a-4876-b69b-7a340dea961c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:40.725583 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4cfedf-c61a-4876-b69b-7a340dea961c None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:35:40.738335 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:40.799764 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e59d3-3162-4aee-a23e-759bcf11d640 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:35:40.800360 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e59d3-3162-4aee-a23e-759bcf11d640 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:40.800631 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e59d3-3162-4aee-a23e-759bcf11d640 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:40.800892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e59d3-3162-4aee-a23e-759bcf11d640 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:40.802483 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7d0e59d3-3162-4aee-a23e-759bcf11d640 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:40.909077 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0e59d3-3162-4aee-a23e-759bcf11d640 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:35:40.910461 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0e59d3-3162-4aee-a23e-759bcf11d640 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:35:40.927711 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 721/1446] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:35:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:35:43.372003 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54453c-3a8e-4d8f-b610-7b7ba49f6af5 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:43.372511 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54453c-3a8e-4d8f-b610-7b7ba49f6af5 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:43.372628 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54453c-3a8e-4d8f-b610-7b7ba49f6af5 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:43.374166 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 726/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 15:35:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:35:43.379414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3134e-9dfb-4735-ab64-e596018e45e4 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:43.379864 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3134e-9dfb-4735-ab64-e596018e45e4 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:43.380460 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3134e-9dfb-4735-ab64-e596018e45e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:43.427818 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:43.446419 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-f8b3134e-9dfb-4735-ab64-e596018e45e4 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:35:43.447619 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-f8b3134e-9dfb-4735-ab64-e596018e45e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:43.543600 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 722/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 15:35:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:35:43.825584 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e54453c-3a8e-4d8f-b610-7b7ba49f6af5 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:35:43.827191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3e54453c-3a8e-4d8f-b610-7b7ba49f6af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:43.931559 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e54453c-3a8e-4d8f-b610-7b7ba49f6af5 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:35:43.972174 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:44.006314 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70803dc6-b4b5-4d81-9c8f-5cbd00bcbebd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:35:44.007008 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70803dc6-b4b5-4d81-9c8f-5cbd00bcbebd service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:44.007424 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70803dc6-b4b5-4d81-9c8f-5cbd00bcbebd service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:44.007742 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70803dc6-b4b5-4d81-9c8f-5cbd00bcbebd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:44.009725 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-70803dc6-b4b5-4d81-9c8f-5cbd00bcbebd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:44.120666 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70803dc6-b4b5-4d81-9c8f-5cbd00bcbebd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:35:44.122072 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70803dc6-b4b5-4d81-9c8f-5cbd00bcbebd service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:35:44.142136 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 727/1449] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:35:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:35:46.511911 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c121e4-9b97-4f3e-966f-5faa2dfcf83c None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:46.512471 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c121e4-9b97-4f3e-966f-5faa2dfcf83c None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:46.512471 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c121e4-9b97-4f3e-966f-5faa2dfcf83c None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:46.513798 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 723/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 15:35:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:35:46.519147 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec4ff1-8db1-4daa-b4e3-a1f14c326ca9 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:46.519552 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec4ff1-8db1-4daa-b4e3-a1f14c326ca9 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:46.519705 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec4ff1-8db1-4daa-b4e3-a1f14c326ca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:46.586227 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-73ec4ff1-8db1-4daa-b4e3-a1f14c326ca9 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:35:46.587422 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-73ec4ff1-8db1-4daa-b4e3-a1f14c326ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:46.607299 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:46.681970 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 728/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 15:35:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:35:46.967444 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c121e4-9b97-4f3e-966f-5faa2dfcf83c None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:35:46.968972 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-15c121e4-9b97-4f3e-966f-5faa2dfcf83c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:47.072498 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c121e4-9b97-4f3e-966f-5faa2dfcf83c None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:35:47.091807 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:47.145163 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d703aff-1d97-4300-984c-1707e7efb10f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:35:47.145754 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d703aff-1d97-4300-984c-1707e7efb10f service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:47.146053 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d703aff-1d97-4300-984c-1707e7efb10f service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:47.146321 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d703aff-1d97-4300-984c-1707e7efb10f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:47.147872 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-6d703aff-1d97-4300-984c-1707e7efb10f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:47.254988 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d703aff-1d97-4300-984c-1707e7efb10f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:35:47.256331 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d703aff-1d97-4300-984c-1707e7efb10f service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:35:47.274650 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 724/1452] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:35:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:35:49.462318 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab079a3e-1544-48b1-8ad4-c19f8f6319b3 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:49.462884 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab079a3e-1544-48b1-8ad4-c19f8f6319b3 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:49.462884 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab079a3e-1544-48b1-8ad4-c19f8f6319b3 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:49.464210 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 729/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 15:35:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:35:49.469579 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26746d-1c8a-41f4-a9b0-c61dc6bc94c1 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:49.470027 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26746d-1c8a-41f4-a9b0-c61dc6bc94c1 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:49.470129 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26746d-1c8a-41f4-a9b0-c61dc6bc94c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:49.534754 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:49.537690 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-7e26746d-1c8a-41f4-a9b0-c61dc6bc94c1 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:35:49.539497 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7e26746d-1c8a-41f4-a9b0-c61dc6bc94c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:49.633519 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 725/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 15:35:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:35:49.918138 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab079a3e-1544-48b1-8ad4-c19f8f6319b3 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:35:49.919679 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ab079a3e-1544-48b1-8ad4-c19f8f6319b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:50.023957 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab079a3e-1544-48b1-8ad4-c19f8f6319b3 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:35:50.071676 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:50.096102 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4874de2-3e7e-4547-9a10-2ce41980abd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:35:50.096699 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4874de2-3e7e-4547-9a10-2ce41980abd3 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:50.096981 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4874de2-3e7e-4547-9a10-2ce41980abd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:50.097256 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4874de2-3e7e-4547-9a10-2ce41980abd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:50.098882 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c4874de2-3e7e-4547-9a10-2ce41980abd3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:50.212694 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4874de2-3e7e-4547-9a10-2ce41980abd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:35:50.214072 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4874de2-3e7e-4547-9a10-2ce41980abd3 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:35:50.231585 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 730/1455] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:35:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:35:52.436001 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320fc1ab-15b6-49d3-b079-e5949a8d8634 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:52.436593 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320fc1ab-15b6-49d3-b079-e5949a8d8634 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:52.436593 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320fc1ab-15b6-49d3-b079-e5949a8d8634 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:52.437896 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 726/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 15:35:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:35:52.443690 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a75501-4abe-4dac-9636-bfbfd174b05a None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:52.444155 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a75501-4abe-4dac-9636-bfbfd174b05a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:52.444250 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a75501-4abe-4dac-9636-bfbfd174b05a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:52.510160 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-e4a75501-4abe-4dac-9636-bfbfd174b05a None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:35:52.511343 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e4a75501-4abe-4dac-9636-bfbfd174b05a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:52.541123 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:52.607471 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 731/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 15:35:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:35:52.889344 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320fc1ab-15b6-49d3-b079-e5949a8d8634 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:35:52.890900 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-320fc1ab-15b6-49d3-b079-e5949a8d8634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:52.993528 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320fc1ab-15b6-49d3-b079-e5949a8d8634 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:35:53.023017 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:53.067270 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052483c9-764f-4aeb-8bde-da2f5280fae7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:35:53.067867 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052483c9-764f-4aeb-8bde-da2f5280fae7 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:53.068141 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052483c9-764f-4aeb-8bde-da2f5280fae7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:53.068409 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052483c9-764f-4aeb-8bde-da2f5280fae7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:53.070018 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-052483c9-764f-4aeb-8bde-da2f5280fae7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:53.177101 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052483c9-764f-4aeb-8bde-da2f5280fae7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:35:53.178483 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052483c9-764f-4aeb-8bde-da2f5280fae7 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:35:53.196569 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 727/1458] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:35:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:35:55.475438 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa79959-0df9-4ad8-a056-64bab17f5bb6 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:55.476038 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa79959-0df9-4ad8-a056-64bab17f5bb6 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:55.476038 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa79959-0df9-4ad8-a056-64bab17f5bb6 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:55.477331 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 732/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 15:35:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:35:55.482731 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd810c09-f8a4-4272-ac9d-d764d6e5c865 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:55.483136 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd810c09-f8a4-4272-ac9d-d764d6e5c865 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:55.483279 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd810c09-f8a4-4272-ac9d-d764d6e5c865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:55.551805 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-bd810c09-f8a4-4272-ac9d-d764d6e5c865 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:35:55.553018 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-bd810c09-f8a4-4272-ac9d-d764d6e5c865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:55.561197 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:55.649838 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 728/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 15:35:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:35:55.930733 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa79959-0df9-4ad8-a056-64bab17f5bb6 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:35:55.932259 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-cfa79959-0df9-4ad8-a056-64bab17f5bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:56.035683 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa79959-0df9-4ad8-a056-64bab17f5bb6 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:35:56.090249 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:56.107808 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bf9dfc-930e-471e-9078-3cf3b3ffbdf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:35:56.108408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf9dfc-930e-471e-9078-3cf3b3ffbdf8 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:56.108685 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf9dfc-930e-471e-9078-3cf3b3ffbdf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:56.108956 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf9dfc-930e-471e-9078-3cf3b3ffbdf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:56.110579 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d9bf9dfc-930e-471e-9078-3cf3b3ffbdf8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:56.218779 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bf9dfc-930e-471e-9078-3cf3b3ffbdf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:35:56.220123 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bf9dfc-930e-471e-9078-3cf3b3ffbdf8 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:35:56.238764 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 733/1461] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:35:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:35:58.446547 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dceb6c7-2b97-4be0-99ac-f5ea7358f874 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:58.447076 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dceb6c7-2b97-4be0-99ac-f5ea7358f874 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:58.447076 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dceb6c7-2b97-4be0-99ac-f5ea7358f874 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:58.448322 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 729/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 15:35:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:35:58.453500 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e63c8d-b5cf-4969-80ec-37d9441e1bba None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:58.453929 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e63c8d-b5cf-4969-80ec-37d9441e1bba None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:58.454086 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e63c8d-b5cf-4969-80ec-37d9441e1bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:58.525334 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d5e63c8d-b5cf-4969-80ec-37d9441e1bba None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:35:58.526594 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d5e63c8d-b5cf-4969-80ec-37d9441e1bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:58.564878 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:58.623356 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 734/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 15:35:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:35:58.909892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dceb6c7-2b97-4be0-99ac-f5ea7358f874 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:35:58.911439 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-0dceb6c7-2b97-4be0-99ac-f5ea7358f874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:59.013521 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dceb6c7-2b97-4be0-99ac-f5ea7358f874 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:35:59.051008 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:35:59.085985 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0306d2d-44d2-4682-afaf-c9ad694e6eb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:35:59.086580 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0306d2d-44d2-4682-afaf-c9ad694e6eb1 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:35:59.086852 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0306d2d-44d2-4682-afaf-c9ad694e6eb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:35:59.087147 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0306d2d-44d2-4682-afaf-c9ad694e6eb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:35:59.088697 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c0306d2d-44d2-4682-afaf-c9ad694e6eb1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:35:59.194021 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0306d2d-44d2-4682-afaf-c9ad694e6eb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:35:59.195438 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0306d2d-44d2-4682-afaf-c9ad694e6eb1 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:35:59.212839 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 730/1464] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:35:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:36:01.391920 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5bbcc9-b523-429f-bdec-6f9516dcab64 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:01.392460 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5bbcc9-b523-429f-bdec-6f9516dcab64 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:01.392753 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5bbcc9-b523-429f-bdec-6f9516dcab64 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:01.394565 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 735/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 15:36:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:36:01.399998 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c365d8e-fbe8-4197-9e7b-fe712bd7cbac None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:01.400414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c365d8e-fbe8-4197-9e7b-fe712bd7cbac None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:01.400538 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c365d8e-fbe8-4197-9e7b-fe712bd7cbac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:01.465442 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-7c365d8e-fbe8-4197-9e7b-fe712bd7cbac None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:36:01.466927 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7c365d8e-fbe8-4197-9e7b-fe712bd7cbac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:01.484671 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:01.562421 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 731/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 15:36:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:36:01.844296 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5bbcc9-b523-429f-bdec-6f9516dcab64 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:36:01.846482 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ed5bbcc9-b523-429f-bdec-6f9516dcab64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:01.946672 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5bbcc9-b523-429f-bdec-6f9516dcab64 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:36:02.015190 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:02.023457 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f481392-c799-4237-a52d-6c874b214865 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:36:02.024061 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f481392-c799-4237-a52d-6c874b214865 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:02.024339 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f481392-c799-4237-a52d-6c874b214865 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:02.024612 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f481392-c799-4237-a52d-6c874b214865 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:02.026325 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7f481392-c799-4237-a52d-6c874b214865 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:02.133976 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f481392-c799-4237-a52d-6c874b214865 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:36:02.135340 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f481392-c799-4237-a52d-6c874b214865 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:36:02.153301 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 736/1467] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:36:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:36:04.307410 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76861ea4-75df-43cf-82c0-abb30fa3685d None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:04.307835 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76861ea4-75df-43cf-82c0-abb30fa3685d None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:04.307966 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76861ea4-75df-43cf-82c0-abb30fa3685d None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:04.309266 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 732/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 15:36:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:36:04.315019 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b007e12-2e54-4d53-9e23-6541c96d240c None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:04.315419 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b007e12-2e54-4d53-9e23-6541c96d240c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:04.315568 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b007e12-2e54-4d53-9e23-6541c96d240c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:04.380337 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-8b007e12-2e54-4d53-9e23-6541c96d240c None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:36:04.381533 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8b007e12-2e54-4d53-9e23-6541c96d240c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:04.428441 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:04.476330 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 737/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 15:36:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:36:04.762849 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76861ea4-75df-43cf-82c0-abb30fa3685d None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:36:04.764364 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-76861ea4-75df-43cf-82c0-abb30fa3685d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:04.869075 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76861ea4-75df-43cf-82c0-abb30fa3685d None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:36:04.915789 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:04.941550 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc80946a-1589-407e-a781-49ae9985c6e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:36:04.942239 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80946a-1589-407e-a781-49ae9985c6e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:04.942539 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80946a-1589-407e-a781-49ae9985c6e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:04.942810 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80946a-1589-407e-a781-49ae9985c6e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:04.944381 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-fc80946a-1589-407e-a781-49ae9985c6e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:05.051218 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc80946a-1589-407e-a781-49ae9985c6e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:36:05.052663 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc80946a-1589-407e-a781-49ae9985c6e8 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:36:05.070098 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 733/1470] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:36:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:36:14.474674 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=22 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:15.067818 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=66 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:25.054827 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca668af-941c-4350-a99b-f6af776736db None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:25.055737 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca668af-941c-4350-a99b-f6af776736db None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:25.055737 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca668af-941c-4350-a99b-f6af776736db None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:25.056725 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 738/1471] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:36:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:36:25.062403 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb2df4-2680-4c2b-abdf-35a7364ca1d7 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:25.062932 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb2df4-2680-4c2b-abdf-35a7364ca1d7 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:25.062932 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb2df4-2680-4c2b-abdf-35a7364ca1d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:25.129330 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-d2cb2df4-2680-4c2b-abdf-35a7364ca1d7 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:36:25.130563 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d2cb2df4-2680-4c2b-abdf-35a7364ca1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:25.226797 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 734/1472] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:36:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:36:25.534279 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c68cf-d4d0-49e0-9514-0c0087a03c07 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:25.534560 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c68cf-d4d0-49e0-9514-0c0087a03c07 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:25.534834 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c68cf-d4d0-49e0-9514-0c0087a03c07 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:25.536154 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 739/1473] 10.222.0.52 () {54 vars in 962 bytes} [Wed Sep 28 15:36:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:36:25.541052 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5279d8-58b1-40c6-9a77-037b8f51bef0 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:25.541336 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5279d8-58b1-40c6-9a77-037b8f51bef0 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:25.541611 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5279d8-58b1-40c6-9a77-037b8f51bef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:25.617375 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-fc5279d8-58b1-40c6-9a77-037b8f51bef0 None None] MFA Rules not processed for user `c6a46bbd249c4434ad7d9442754f098a`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:36:25.618621 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-fc5279d8-58b1-40c6-9a77-037b8f51bef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:25.638897 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:25.694980 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 735/1474] 10.222.0.52 () {58 vars in 1059 bytes} [Wed Sep 28 15:36:25 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:36:25.702727 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78c68cf-d4d0-49e0-9514-0c0087a03c07 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:36:25.704262 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c78c68cf-d4d0-49e0-9514-0c0087a03c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:25.807874 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78c68cf-d4d0-49e0-9514-0c0087a03c07 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:36:25.880829 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8294bb-e96f-430d-bb94-50bdf62bb39a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6a46bbd249c4434ad7d9442754f098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:36:25.881427 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8294bb-e96f-430d-bb94-50bdf62bb39a service glance] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:25.881705 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8294bb-e96f-430d-bb94-50bdf62bb39a service glance] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:25.882010 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8294bb-e96f-430d-bb94-50bdf62bb39a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:25.883590 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8f8294bb-e96f-430d-bb94-50bdf62bb39a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:25.936821 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:25.990978 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8294bb-e96f-430d-bb94-50bdf62bb39a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:36:25.992410 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8294bb-e96f-430d-bb94-50bdf62bb39a service glance] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:36:26.011675 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 740/1475] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:36:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:36:28.259651 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539e2d2-423b-4c5c-8675-466fb849611d None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:28.260197 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539e2d2-423b-4c5c-8675-466fb849611d None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:28.260197 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539e2d2-423b-4c5c-8675-466fb849611d None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:28.261731 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 736/1476] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:36:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:36:28.266756 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7ce59-bc30-46ec-a620-42f238e28ab6 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:28.267143 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7ce59-bc30-46ec-a620-42f238e28ab6 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:28.267309 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7ce59-bc30-46ec-a620-42f238e28ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:28.332418 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-45f7ce59-bc30-46ec-a620-42f238e28ab6 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:36:28.333607 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-45f7ce59-bc30-46ec-a620-42f238e28ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:28.428352 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 741/1477] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:36:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:36:28.729182 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8539e2d2-423b-4c5c-8675-466fb849611d None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:36:28.730729 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-8539e2d2-423b-4c5c-8675-466fb849611d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:28.856614 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8539e2d2-423b-4c5c-8675-466fb849611d None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:36:28.906260 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:28.950040 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad455c-3e12-42c0-bdcd-472aa482eb1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:36:28.950667 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad455c-3e12-42c0-bdcd-472aa482eb1a admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:28.950944 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad455c-3e12-42c0-bdcd-472aa482eb1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:28.951212 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad455c-3e12-42c0-bdcd-472aa482eb1a admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:28.951839 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aad455c-3e12-42c0-bdcd-472aa482eb1a admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:36:28.953578 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: WARNING py.warnings [None req-3aad455c-3e12-42c0-bdcd-472aa482eb1a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 15:36:28.953578 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: warnings.warn(msg) Sep 28 15:36:28.953578 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: Sep 28 15:36:28.954750 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aad455c-3e12-42c0-bdcd-472aa482eb1a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:36:28.974015 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 737/1478] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Sep 28 15:36:28 2022] GET /identity/v3/projects => generated 2553 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:36:31.147271 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ec4e8-a7b4-45af-92d7-be2be33b415a None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:31.147840 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ec4e8-a7b4-45af-92d7-be2be33b415a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:31.147840 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ec4e8-a7b4-45af-92d7-be2be33b415a None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:31.149150 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 742/1479] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:36:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:36:31.154539 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd18d9-76a9-41ff-b978-4c860c6f0c0b None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:31.154942 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd18d9-76a9-41ff-b978-4c860c6f0c0b None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:31.155108 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd18d9-76a9-41ff-b978-4c860c6f0c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:31.219991 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-27dd18d9-76a9-41ff-b978-4c860c6f0c0b None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:36:31.221803 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-27dd18d9-76a9-41ff-b978-4c860c6f0c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:31.317942 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 738/1480] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:36:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:36:31.611484 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9ec4e8-a7b4-45af-92d7-be2be33b415a None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:36:31.613149 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-fd9ec4e8-a7b4-45af-92d7-be2be33b415a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:31.622780 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:31.756538 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9ec4e8-a7b4-45af-92d7-be2be33b415a None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:36:31.833532 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:31.866960 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251e43a5-400c-4069-a889-742b06e46025 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf14b64bf04b2ba74d2fe1fd41ff71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:36:31.867565 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251e43a5-400c-4069-a889-742b06e46025 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:31.867848 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251e43a5-400c-4069-a889-742b06e46025 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:31.868124 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251e43a5-400c-4069-a889-742b06e46025 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:31.869722 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-251e43a5-400c-4069-a889-742b06e46025 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:31.959146 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251e43a5-400c-4069-a889-742b06e46025 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:36:31.960727 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251e43a5-400c-4069-a889-742b06e46025 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:36:31.978663 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 743/1481] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:36:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:36:34.312585 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004878b-1eb0-4652-a9c7-55735163473d None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:34.313084 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004878b-1eb0-4652-a9c7-55735163473d None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:34.313196 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004878b-1eb0-4652-a9c7-55735163473d None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:34.315066 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 739/1482] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:36:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:36:34.320636 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9da33-b2f2-414b-a461-bd9f2d3f8916 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:34.321081 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9da33-b2f2-414b-a461-bd9f2d3f8916 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:34.321191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b9da33-b2f2-414b-a461-bd9f2d3f8916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:34.385858 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-b5b9da33-b2f2-414b-a461-bd9f2d3f8916 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:36:34.387061 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b5b9da33-b2f2-414b-a461-bd9f2d3f8916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:34.431475 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:34.480686 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 744/1483] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:36:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:36:34.766395 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d004878b-1eb0-4652-a9c7-55735163473d None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:36:34.767982 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d004878b-1eb0-4652-a9c7-55735163473d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:34.793694 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:34.881982 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d004878b-1eb0-4652-a9c7-55735163473d None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:36:34.952881 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c74156-9745-49f0-b1ca-386074182a0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:36:34.953477 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c74156-9745-49f0-b1ca-386074182a0e service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:34.953750 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c74156-9745-49f0-b1ca-386074182a0e service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:34.954047 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c74156-9745-49f0-b1ca-386074182a0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:34.955658 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-f4c74156-9745-49f0-b1ca-386074182a0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:35.027579 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:35.062815 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c74156-9745-49f0-b1ca-386074182a0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:36:35.064239 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c74156-9745-49f0-b1ca-386074182a0e service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:36:35.081734 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 740/1484] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:36:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:36:37.267212 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe2eff-1f6a-4140-9319-5af88e3282ae None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:37.268308 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe2eff-1f6a-4140-9319-5af88e3282ae None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:37.268413 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe2eff-1f6a-4140-9319-5af88e3282ae None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:37.269753 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 745/1485] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:36:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:36:37.274965 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488cfad8-878d-46b1-84e3-570340f50564 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:37.275415 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488cfad8-878d-46b1-84e3-570340f50564 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:37.275511 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488cfad8-878d-46b1-84e3-570340f50564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:37.339809 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-488cfad8-878d-46b1-84e3-570340f50564 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:36:37.340997 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-488cfad8-878d-46b1-84e3-570340f50564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:37.438814 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 741/1486] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:36:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:36:37.728858 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe2eff-1f6a-4140-9319-5af88e3282ae None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:36:37.730478 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-28fe2eff-1f6a-4140-9319-5af88e3282ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:37.849372 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe2eff-1f6a-4140-9319-5af88e3282ae None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:36:37.921554 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:37.923236 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d143ad81-5571-4d2f-89e3-3d738fd32c82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:36:37.924577 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d143ad81-5571-4d2f-89e3-3d738fd32c82 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:37.925096 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d143ad81-5571-4d2f-89e3-3d738fd32c82 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:37.925528 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d143ad81-5571-4d2f-89e3-3d738fd32c82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:37.927397 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d143ad81-5571-4d2f-89e3-3d738fd32c82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:38.033412 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d143ad81-5571-4d2f-89e3-3d738fd32c82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:36:38.034862 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d143ad81-5571-4d2f-89e3-3d738fd32c82 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:36:38.052756 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 746/1487] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:36:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:36:40.352569 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52843afc-c245-4bbb-ac45-647c2b61265c None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:40.353070 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52843afc-c245-4bbb-ac45-647c2b61265c None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:40.353167 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52843afc-c245-4bbb-ac45-647c2b61265c None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:40.354456 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 742/1488] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:36:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:36:40.359807 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2542e55b-f296-4e85-83ea-f9cd2826776f None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:40.360247 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2542e55b-f296-4e85-83ea-f9cd2826776f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:40.360357 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2542e55b-f296-4e85-83ea-f9cd2826776f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:40.562502 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-2542e55b-f296-4e85-83ea-f9cd2826776f None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:36:40.563816 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2542e55b-f296-4e85-83ea-f9cd2826776f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:40.615811 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:40.658928 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 747/1489] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:36:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:36:40.960010 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52843afc-c245-4bbb-ac45-647c2b61265c None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:36:40.961543 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-52843afc-c245-4bbb-ac45-647c2b61265c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:40.996608 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:41.063894 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52843afc-c245-4bbb-ac45-647c2b61265c None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:36:41.136407 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c178788-8e16-433c-8af3-ff69e2c2c531 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:36:41.137004 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c178788-8e16-433c-8af3-ff69e2c2c531 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:41.137279 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c178788-8e16-433c-8af3-ff69e2c2c531 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:41.137548 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c178788-8e16-433c-8af3-ff69e2c2c531 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:41.139139 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-8c178788-8e16-433c-8af3-ff69e2c2c531 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:41.217559 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:41.246578 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c178788-8e16-433c-8af3-ff69e2c2c531 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:36:41.247988 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c178788-8e16-433c-8af3-ff69e2c2c531 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:36:41.266575 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 743/1490] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:36:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:36:43.619649 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c40b7f-5f26-4d1b-9658-f70f1aa65ab7 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:43.620218 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c40b7f-5f26-4d1b-9658-f70f1aa65ab7 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:43.620218 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c40b7f-5f26-4d1b-9658-f70f1aa65ab7 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:43.621533 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 748/1491] 10.222.0.52 () {54 vars in 927 bytes} [Wed Sep 28 15:36:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:36:43.627240 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb2364-f83f-4212-a9c0-30c1d3b802e6 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:43.627689 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb2364-f83f-4212-a9c0-30c1d3b802e6 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:43.627849 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb2364-f83f-4212-a9c0-30c1d3b802e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:43.692482 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-b2cb2364-f83f-4212-a9c0-30c1d3b802e6 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:36:43.693666 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b2cb2364-f83f-4212-a9c0-30c1d3b802e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:43.789977 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 744/1492] 10.222.0.52 () {58 vars in 1024 bytes} [Wed Sep 28 15:36:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:36:44.079002 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c40b7f-5f26-4d1b-9658-f70f1aa65ab7 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:36:44.081179 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-04c40b7f-5f26-4d1b-9658-f70f1aa65ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:44.191096 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c40b7f-5f26-4d1b-9658-f70f1aa65ab7 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:36:44.262150 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60f260f-41b1-4332-9b24-a0869069f5c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:36:44.262744 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f260f-41b1-4332-9b24-a0869069f5c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:36:44.263019 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f260f-41b1-4332-9b24-a0869069f5c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:36:44.263288 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f260f-41b1-4332-9b24-a0869069f5c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:36:44.264854 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b60f260f-41b1-4332-9b24-a0869069f5c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:36:44.274560 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:44.372852 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60f260f-41b1-4332-9b24-a0869069f5c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:36:44.374252 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60f260f-41b1-4332-9b24-a0869069f5c0 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:36:44.393095 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 749/1493] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:36:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:36:53.788464 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=82 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:36:54.390381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=50 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:38:03.414176 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada0858-2b63-4dc0-8afa-5fe5a50ba4b3 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:38:03.414725 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada0858-2b63-4dc0-8afa-5fe5a50ba4b3 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:38:03.414725 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada0858-2b63-4dc0-8afa-5fe5a50ba4b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:38:03.490975 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-6ada0858-2b63-4dc0-8afa-5fe5a50ba4b3 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:38:03.492161 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-6ada0858-2b63-4dc0-8afa-5fe5a50ba4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:38:03.586378 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 745/1494] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:38:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:38:03.595055 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd6537-0d94-4930-ad27-895a376ca492 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:38:03.595557 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd6537-0d94-4930-ad27-895a376ca492 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:38:03.595670 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd6537-0d94-4930-ad27-895a376ca492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:38:03.670038 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-f9bd6537-0d94-4930-ad27-895a376ca492 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:38:03.671233 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f9bd6537-0d94-4930-ad27-895a376ca492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:38:03.765909 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 750/1495] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:38:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:38:03.771014 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ada0858-2b63-4dc0-8afa-5fe5a50ba4b3 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:38:03.772210 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-6ada0858-2b63-4dc0-8afa-5fe5a50ba4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:38:03.784053 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ada0858-2b63-4dc0-8afa-5fe5a50ba4b3 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:38:03.843556 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:38:03.877148 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e473ce-8157-48a2-b9b7-64824c8e3176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:38:03.877748 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e473ce-8157-48a2-b9b7-64824c8e3176 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:38:03.878059 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e473ce-8157-48a2-b9b7-64824c8e3176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:38:03.878317 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e473ce-8157-48a2-b9b7-64824c8e3176 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:38:03.878948 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e473ce-8157-48a2-b9b7-64824c8e3176 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:38:03.879245 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e473ce-8157-48a2-b9b7-64824c8e3176 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:38:03.880244 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: WARNING py.warnings [None req-21e473ce-8157-48a2-b9b7-64824c8e3176 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 28 15:38:03.880244 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: warnings.warn(msg) Sep 28 15:38:03.880244 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: Sep 28 15:38:03.881129 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e473ce-8157-48a2-b9b7-64824c8e3176 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:38:03.892507 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 746/1496] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:38:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:38:03.900739 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd6537-0d94-4930-ad27-895a376ca492 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:38:03.910432 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd6537-0d94-4930-ad27-895a376ca492 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:38:03.970270 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:38:04.003600 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8347e220-dfff-4752-9006-72f891b7f50b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:38:04.004198 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347e220-dfff-4752-9006-72f891b7f50b admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:38:04.004476 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347e220-dfff-4752-9006-72f891b7f50b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:38:04.004744 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347e220-dfff-4752-9006-72f891b7f50b admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:38:04.005367 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8347e220-dfff-4752-9006-72f891b7f50b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:38:04.007123 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: WARNING py.warnings [None req-8347e220-dfff-4752-9006-72f891b7f50b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 15:38:04.007123 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: warnings.warn(msg) Sep 28 15:38:04.007123 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: Sep 28 15:38:04.008323 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8347e220-dfff-4752-9006-72f891b7f50b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:38:04.063490 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 751/1497] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:38:03 2022] POST /identity/v3/projects => generated 379 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:38:04.067598 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e473ce-8157-48a2-b9b7-64824c8e3176 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:38:04.069080 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-21e473ce-8157-48a2-b9b7-64824c8e3176 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:38:04.102399 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e473ce-8157-48a2-b9b7-64824c8e3176 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:38:04.193479 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea657f3-1d77-46ab-aac3-7e695f4b4a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:38:04.194089 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea657f3-1d77-46ab-aac3-7e695f4b4a7f admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:38:04.194363 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea657f3-1d77-46ab-aac3-7e695f4b4a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:38:04.194630 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea657f3-1d77-46ab-aac3-7e695f4b4a7f admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:38:04.195252 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea657f3-1d77-46ab-aac3-7e695f4b4a7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:38:04.196247 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: WARNING py.warnings [None req-1ea657f3-1d77-46ab-aac3-7e695f4b4a7f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 15:38:04.196247 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: warnings.warn(msg) Sep 28 15:38:04.196247 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: Sep 28 15:38:04.197446 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea657f3-1d77-46ab-aac3-7e695f4b4a7f admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:38:04.239822 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 747/1498] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:38:04 2022] POST /identity/v3/users => generated 290 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:38:04.243921 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8347e220-dfff-4752-9006-72f891b7f50b admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:38:04.245419 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8347e220-dfff-4752-9006-72f891b7f50b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:38:04.308761 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8347e220-dfff-4752-9006-72f891b7f50b admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:38:04.389666 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:38:04.401093 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15401ab0-6545-49ee-aa34-514ef7bbf7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:38:04.401691 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15401ab0-6545-49ee-aa34-514ef7bbf7d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:38:04.401999 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15401ab0-6545-49ee-aa34-514ef7bbf7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:38:04.402268 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15401ab0-6545-49ee-aa34-514ef7bbf7d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:38:04.402890 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15401ab0-6545-49ee-aa34-514ef7bbf7d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:38:04.403884 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: WARNING py.warnings [None req-15401ab0-6545-49ee-aa34-514ef7bbf7d0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 28 15:38:04.403884 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: warnings.warn(msg) Sep 28 15:38:04.403884 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: Sep 28 15:38:04.404757 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15401ab0-6545-49ee-aa34-514ef7bbf7d0 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:38:04.417227 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 752/1499] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:38:04 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:38:04.421235 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea657f3-1d77-46ab-aac3-7e695f4b4a7f admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:38:04.423372 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-1ea657f3-1d77-46ab-aac3-7e695f4b4a7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:38:04.434581 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea657f3-1d77-46ab-aac3-7e695f4b4a7f admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:38:04.460681 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:38:04.532328 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea3126-3a50-4170-8794-e65b7206f70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:38:04.532921 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea3126-3a50-4170-8794-e65b7206f70b admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:38:04.533190 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea3126-3a50-4170-8794-e65b7206f70b admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:38:04.533449 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea3126-3a50-4170-8794-e65b7206f70b admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:38:04.534108 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ea3126-3a50-4170-8794-e65b7206f70b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:38:04.535099 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: WARNING py.warnings [None req-d1ea3126-3a50-4170-8794-e65b7206f70b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 28 15:38:04.535099 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: warnings.warn(msg) Sep 28 15:38:04.535099 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: Sep 28 15:38:04.536007 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ea3126-3a50-4170-8794-e65b7206f70b admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:38:04.548067 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 748/1500] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:38:04 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:38:04.552337 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15401ab0-6545-49ee-aa34-514ef7bbf7d0 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:38:04.553875 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-15401ab0-6545-49ee-aa34-514ef7bbf7d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:38:04.586600 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15401ab0-6545-49ee-aa34-514ef7bbf7d0 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:38:04.677920 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8fcbe6-e789-4d16-9c04-664e33e381bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:38:04.678536 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8fcbe6-e789-4d16-9c04-664e33e381bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:38:04.678774 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8fcbe6-e789-4d16-9c04-664e33e381bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:38:04.679045 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8fcbe6-e789-4d16-9c04-664e33e381bb admin admin] PATH_INFO: `/v3/projects/5e1d1a7914cb4ea9abe1a13f0a33ca82/users/b4e34fcec33a46fdaa7661f79d515def/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:38:04.701657 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8fcbe6-e789-4d16-9c04-664e33e381bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e1d1a7914cb4ea9abe1a13f0a33ca82, user_id=b4e34fcec33a46fdaa7661f79d515def, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:38:04.702976 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: WARNING py.warnings [None req-9b8fcbe6-e789-4d16-9c04-664e33e381bb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 15:38:04.702976 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: warnings.warn(msg) Sep 28 15:38:04.702976 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: Sep 28 15:38:04.705455 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8fcbe6-e789-4d16-9c04-664e33e381bb admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:38:04.732986 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 753/1501] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:38:04 2022] PUT /identity/v3/projects/5e1d1a7914cb4ea9abe1a13f0a33ca82/users/b4e34fcec33a46fdaa7661f79d515def/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:38:04.745401 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c29137-18b6-4e7f-84e4-5398b413e8ab None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:38:04.745682 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c29137-18b6-4e7f-84e4-5398b413e8ab None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:38:04.745984 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c29137-18b6-4e7f-84e4-5398b413e8ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:38:04.802955 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-88c29137-18b6-4e7f-84e4-5398b413e8ab None None] MFA Rules not processed for user `b4e34fcec33a46fdaa7661f79d515def`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:38:04.804421 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-88c29137-18b6-4e7f-84e4-5398b413e8ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:38:04.879510 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 749/1502] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:38:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:38:04.893154 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e55258-9705-4fc9-b6ed-9de58b710b58 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:38:04.893508 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e55258-9705-4fc9-b6ed-9de58b710b58 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:38:04.893784 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e55258-9705-4fc9-b6ed-9de58b710b58 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:38:04.895143 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 754/1503] 10.222.0.52 () {54 vars in 963 bytes} [Wed Sep 28 15:38:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:38:04.899737 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c7b53-ff87-4d42-8fa1-66fa123c9739 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:38:04.900020 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c7b53-ff87-4d42-8fa1-66fa123c9739 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:38:04.900299 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c7b53-ff87-4d42-8fa1-66fa123c9739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:38:04.930828 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:38:04.974624 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-ff7c7b53-ff87-4d42-8fa1-66fa123c9739 None None] MFA Rules not processed for user `01833709177e4e68a2fa52fb9963d3bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:38:04.975809 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ff7c7b53-ff87-4d42-8fa1-66fa123c9739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:38:05.050964 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 750/1504] 10.222.0.52 () {58 vars in 1060 bytes} [Wed Sep 28 15:38:04 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:38:05.058640 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e55258-9705-4fc9-b6ed-9de58b710b58 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:38:05.060134 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-68e55258-9705-4fc9-b6ed-9de58b710b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:38:05.108762 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:38:05.162554 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e55258-9705-4fc9-b6ed-9de58b710b58 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:38:05.232581 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dd3ab0-bd9a-42c3-9f05-df06be542bb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:38:05.233097 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd3ab0-bd9a-42c3-9f05-df06be542bb1 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:38:05.233373 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd3ab0-bd9a-42c3-9f05-df06be542bb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:38:05.233639 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd3ab0-bd9a-42c3-9f05-df06be542bb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:38:05.236046 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-83dd3ab0-bd9a-42c3-9f05-df06be542bb1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:38:05.337221 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dd3ab0-bd9a-42c3-9f05-df06be542bb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:38:05.338678 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dd3ab0-bd9a-42c3-9f05-df06be542bb1 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:38:05.342096 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:38:05.356086 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 755/1505] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:38:05 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:38:05.402591 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f348e6e4-e29e-4c80-87dc-4f1b2d67d20b None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:38:05.402878 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f348e6e4-e29e-4c80-87dc-4f1b2d67d20b None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:38:05.403179 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f348e6e4-e29e-4c80-87dc-4f1b2d67d20b None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:38:05.404414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 751/1506] 10.222.0.52 () {52 vars in 830 bytes} [Wed Sep 28 15:38:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:38:05.407901 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dd3ab0-bd9a-42c3-9f05-df06be542bb1 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:38:05.409093 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-83dd3ab0-bd9a-42c3-9f05-df06be542bb1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:38:05.506275 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dd3ab0-bd9a-42c3-9f05-df06be542bb1 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:38:05.596407 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116bceb9-af26-4073-804f-a02f05b151e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:38:05.597028 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116bceb9-af26-4073-804f-a02f05b151e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:38:05.597302 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116bceb9-af26-4073-804f-a02f05b151e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:38:05.597570 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116bceb9-af26-4073-804f-a02f05b151e1 admin admin] PATH_INFO: `/v3/users/b4e34fcec33a46fdaa7661f79d515def` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:38:05.610577 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116bceb9-af26-4073-804f-a02f05b151e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4e34fcec33a46fdaa7661f79d515def)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:38:05.611587 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: WARNING py.warnings [None req-116bceb9-af26-4073-804f-a02f05b151e1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 15:38:05.611587 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: warnings.warn(msg) Sep 28 15:38:05.611587 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: Sep 28 15:38:05.612793 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116bceb9-af26-4073-804f-a02f05b151e1 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:38:05.622284 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:38:05.678633 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-116bceb9-af26-4073-804f-a02f05b151e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4e34fcec33a46fdaa7661f79d515def'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:38:05.679278 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-116bceb9-af26-4073-804f-a02f05b151e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4e34fcec33a46fdaa7661f79d515def'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:38:05.702795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-116bceb9-af26-4073-804f-a02f05b151e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4e34fcec33a46fdaa7661f79d515def'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:38:05.711982 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-116bceb9-af26-4073-804f-a02f05b151e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4e34fcec33a46fdaa7661f79d515def'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:38:05.726369 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-116bceb9-af26-4073-804f-a02f05b151e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4e34fcec33a46fdaa7661f79d515def'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:38:05.732672 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 756/1507] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:38:05 2022] DELETE /identity/v3/users/b4e34fcec33a46fdaa7661f79d515def => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:38:05.739475 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046999e-3b37-4aae-bfea-334f4fe0fba8 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:38:05.739958 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046999e-3b37-4aae-bfea-334f4fe0fba8 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:38:05.740121 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a046999e-3b37-4aae-bfea-334f4fe0fba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:38:05.797141 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-a046999e-3b37-4aae-bfea-334f4fe0fba8 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:38:05.798879 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a046999e-3b37-4aae-bfea-334f4fe0fba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:38:05.873506 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:38:05.886766 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 752/1508] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:38:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:38:05.896311 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116bceb9-af26-4073-804f-a02f05b151e1 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:38:05.897518 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-116bceb9-af26-4073-804f-a02f05b151e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:38:05.998496 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116bceb9-af26-4073-804f-a02f05b151e1 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:38:06.052834 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:38:06.072935 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19d2d62-94b5-44ba-95be-8189ccf3d4fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:38:06.073536 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d2d62-94b5-44ba-95be-8189ccf3d4fe service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:38:06.073814 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d2d62-94b5-44ba-95be-8189ccf3d4fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:38:06.074113 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d2d62-94b5-44ba-95be-8189ccf3d4fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:38:06.075693 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f19d2d62-94b5-44ba-95be-8189ccf3d4fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:38:06.181773 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19d2d62-94b5-44ba-95be-8189ccf3d4fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:38:06.183225 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19d2d62-94b5-44ba-95be-8189ccf3d4fe service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:38:06.201006 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 757/1509] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:38:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:38:06.563678 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a046999e-3b37-4aae-bfea-334f4fe0fba8 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:38:06.564898 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a046999e-3b37-4aae-bfea-334f4fe0fba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:38:06.576226 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a046999e-3b37-4aae-bfea-334f4fe0fba8 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:38:06.676977 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b6aa6-ea2b-4d63-b20d-69bd5af252ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:38:06.677603 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b6aa6-ea2b-4d63-b20d-69bd5af252ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:38:06.677903 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b6aa6-ea2b-4d63-b20d-69bd5af252ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:38:06.678177 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b6aa6-ea2b-4d63-b20d-69bd5af252ac admin admin] PATH_INFO: `/v3/projects/5e1d1a7914cb4ea9abe1a13f0a33ca82` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:38:06.689253 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0b6aa6-ea2b-4d63-b20d-69bd5af252ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e1d1a7914cb4ea9abe1a13f0a33ca82)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:38:06.690298 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: WARNING py.warnings [None req-6c0b6aa6-ea2b-4d63-b20d-69bd5af252ac admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 15:38:06.690298 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: warnings.warn(msg) Sep 28 15:38:06.690298 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: Sep 28 15:38:06.691491 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0b6aa6-ea2b-4d63-b20d-69bd5af252ac admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:38:06.751302 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-6c0b6aa6-ea2b-4d63-b20d-69bd5af252ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e1d1a7914cb4ea9abe1a13f0a33ca82', 'request_id': 'req-6c0b6aa6-ea2b-4d63-b20d-69bd5af252ac'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:38:06.761285 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-6c0b6aa6-ea2b-4d63-b20d-69bd5af252ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e1d1a7914cb4ea9abe1a13f0a33ca82', 'request_id': 'req-6c0b6aa6-ea2b-4d63-b20d-69bd5af252ac'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:38:06.774384 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-6c0b6aa6-ea2b-4d63-b20d-69bd5af252ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e1d1a7914cb4ea9abe1a13f0a33ca82', 'request_id': 'req-6c0b6aa6-ea2b-4d63-b20d-69bd5af252ac'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:38:06.798526 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-6c0b6aa6-ea2b-4d63-b20d-69bd5af252ac admin admin] The token cache is being invalidate because project 5e1d1a7914cb4ea9abe1a13f0a33ca82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:38:06.798814 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-6c0b6aa6-ea2b-4d63-b20d-69bd5af252ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:38:06.799242 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-6c0b6aa6-ea2b-4d63-b20d-69bd5af252ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:38:06.800693 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 753/1510] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:38:06 2022] DELETE /identity/v3/projects/5e1d1a7914cb4ea9abe1a13f0a33ca82 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:38:16.198861 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=62 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:38:16.795798 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:39:29.723188 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e43c4-8bc8-47aa-bbfd-34f8d34e4c38 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:39:29.723859 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e43c4-8bc8-47aa-bbfd-34f8d34e4c38 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:39:29.724035 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e43c4-8bc8-47aa-bbfd-34f8d34e4c38 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:39:29.725301 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 758/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 15:39:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:39:29.730691 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f68b9-b4e7-4dcc-9226-472945e500f5 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:39:29.731229 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f68b9-b4e7-4dcc-9226-472945e500f5 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:39:29.731229 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f68b9-b4e7-4dcc-9226-472945e500f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:39:29.814370 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-2b3f68b9-b4e7-4dcc-9226-472945e500f5 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:39:29.815561 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-2b3f68b9-b4e7-4dcc-9226-472945e500f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:39:29.917006 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 754/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 15:39:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:39:30.074217 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238e43c4-8bc8-47aa-bbfd-34f8d34e4c38 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:39:30.076431 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-238e43c4-8bc8-47aa-bbfd-34f8d34e4c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:39:30.198639 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238e43c4-8bc8-47aa-bbfd-34f8d34e4c38 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:39:30.204826 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b23e6-5e2f-4fa6-b838-f94ff19af9a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf14b64bf04b2ba74d2fe1fd41ff71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:39:30.205432 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b23e6-5e2f-4fa6-b838-f94ff19af9a5 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:39:30.205713 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b23e6-5e2f-4fa6-b838-f94ff19af9a5 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:39:30.206027 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b23e6-5e2f-4fa6-b838-f94ff19af9a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:39:30.214622 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b23e6-5e2f-4fa6-b838-f94ff19af9a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:39:30.216219 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b23e6-5e2f-4fa6-b838-f94ff19af9a5 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:39:30.226088 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 759/1513] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:39:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:39:31.302594 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0eaf9b-5402-46df-a49a-071faa454293 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:39:31.303071 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0eaf9b-5402-46df-a49a-071faa454293 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:39:31.303171 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0eaf9b-5402-46df-a49a-071faa454293 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:39:31.304460 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 755/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 15:39:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:39:31.310562 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9cc4da-a9f3-44fa-a336-e4d79ca75165 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:39:31.311023 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9cc4da-a9f3-44fa-a336-e4d79ca75165 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:39:31.311136 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9cc4da-a9f3-44fa-a336-e4d79ca75165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:39:31.331520 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-ab9cc4da-a9f3-44fa-a336-e4d79ca75165 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:39:31.332712 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ab9cc4da-a9f3-44fa-a336-e4d79ca75165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:39:31.340484 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 760/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 15:39:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:39:31.491219 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0eaf9b-5402-46df-a49a-071faa454293 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:39:31.506684 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0eaf9b-5402-46df-a49a-071faa454293 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:39:31.512030 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee55086-20f0-444b-be04-b331769f78ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf14b64bf04b2ba74d2fe1fd41ff71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:39:31.512628 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee55086-20f0-444b-be04-b331769f78ad service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:39:31.512901 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee55086-20f0-444b-be04-b331769f78ad service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:39:31.513167 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee55086-20f0-444b-be04-b331769f78ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:39:31.521643 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee55086-20f0-444b-be04-b331769f78ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:39:31.523243 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee55086-20f0-444b-be04-b331769f78ad service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:39:31.532905 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 756/1516] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:39:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:39:31.562904 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef600817-b014-4e14-8f53-00b38257a77a None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:39:31.563184 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef600817-b014-4e14-8f53-00b38257a77a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:39:31.563455 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef600817-b014-4e14-8f53-00b38257a77a None None] PATH_INFO: `/v3` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:39:31.564725 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 761/1517] 10.222.0.50 () {54 vars in 925 bytes} [Wed Sep 28 15:39:31 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:39:31.571213 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc869f7-da0f-44e5-8a6d-2cae9b7ddb05 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:39:31.571501 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc869f7-da0f-44e5-8a6d-2cae9b7ddb05 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:39:31.571776 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc869f7-da0f-44e5-8a6d-2cae9b7ddb05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:39:31.641475 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-5fc869f7-da0f-44e5-8a6d-2cae9b7ddb05 None None] MFA Rules not processed for user `de934d0105bd4d01ba6c06a4a0d36704`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:39:31.642740 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-5fc869f7-da0f-44e5-8a6d-2cae9b7ddb05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:39:31.662863 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:39:31.719809 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 757/1518] 10.222.0.50 () {58 vars in 1015 bytes} [Wed Sep 28 15:39:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:39:35.293025 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d97c3-5c0e-45d4-b30a-bbffc0b06d5f None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:39:35.293517 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d97c3-5c0e-45d4-b30a-bbffc0b06d5f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:39:35.293612 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d97c3-5c0e-45d4-b30a-bbffc0b06d5f None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:39:35.295094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 762/1519] 10.222.0.52 () {54 vars in 961 bytes} [Wed Sep 28 15:39:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:39:35.300757 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0ef7d-d5a8-4193-b756-665c1426a795 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:39:35.301181 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0ef7d-d5a8-4193-b756-665c1426a795 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:39:35.301289 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0ef7d-d5a8-4193-b756-665c1426a795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:39:35.321259 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-eaa0ef7d-d5a8-4193-b756-665c1426a795 None None] MFA Rules not processed for user `de934d0105bd4d01ba6c06a4a0d36704`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:39:35.322491 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-eaa0ef7d-d5a8-4193-b756-665c1426a795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:39:35.331267 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 758/1520] 10.222.0.52 () {58 vars in 1058 bytes} [Wed Sep 28 15:39:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:39:35.338719 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2d97c3-5c0e-45d4-b30a-bbffc0b06d5f None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:39:35.351802 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2d97c3-5c0e-45d4-b30a-bbffc0b06d5f None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:39:35.356619 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db63f548-a276-4ac3-9f1b-8d095eed0299 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de934d0105bd4d01ba6c06a4a0d36704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:39:35.357210 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db63f548-a276-4ac3-9f1b-8d095eed0299 service placement] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:39:35.357515 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db63f548-a276-4ac3-9f1b-8d095eed0299 service placement] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:39:35.357783 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db63f548-a276-4ac3-9f1b-8d095eed0299 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:39:35.366252 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db63f548-a276-4ac3-9f1b-8d095eed0299 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:39:35.367817 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db63f548-a276-4ac3-9f1b-8d095eed0299 service placement] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:39:35.378033 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 763/1521] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:39:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:39:42.990397 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d69ba-5c2b-4c52-b0bd-6df798519cf1 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:39:42.990991 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d69ba-5c2b-4c52-b0bd-6df798519cf1 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:39:42.990991 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d69ba-5c2b-4c52-b0bd-6df798519cf1 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:39:42.992262 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 759/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 15:39:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:39:42.998326 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379b663-b6e6-4fda-a133-dcc259348289 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:39:42.999322 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379b663-b6e6-4fda-a133-dcc259348289 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:39:42.999521 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379b663-b6e6-4fda-a133-dcc259348289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:39:43.018633 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-8379b663-b6e6-4fda-a133-dcc259348289 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:39:43.019799 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8379b663-b6e6-4fda-a133-dcc259348289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:39:43.027407 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 764/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 15:39:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:39:43.179928 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d69ba-5c2b-4c52-b0bd-6df798519cf1 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:39:43.194157 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d69ba-5c2b-4c52-b0bd-6df798519cf1 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:39:43.199482 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d5ccd-b081-4ca6-9426-33532d960050 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf14b64bf04b2ba74d2fe1fd41ff71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:39:43.200064 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d5ccd-b081-4ca6-9426-33532d960050 service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:39:43.200339 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d5ccd-b081-4ca6-9426-33532d960050 service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:39:43.200604 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d5ccd-b081-4ca6-9426-33532d960050 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:39:43.208673 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0d5ccd-b081-4ca6-9426-33532d960050 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:39:43.210275 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0d5ccd-b081-4ca6-9426-33532d960050 service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:39:43.220105 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 760/1524] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:39:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:39:44.287561 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44c24e-c75d-4bb3-a524-8e0221831133 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:39:44.288125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44c24e-c75d-4bb3-a524-8e0221831133 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:39:44.288125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44c24e-c75d-4bb3-a524-8e0221831133 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:39:44.289626 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 765/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 28 15:39:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:39:44.296475 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127f0ae0-a7f7-4c28-b69a-6874c376efef None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:39:44.296907 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127f0ae0-a7f7-4c28-b69a-6874c376efef None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:39:44.297003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127f0ae0-a7f7-4c28-b69a-6874c376efef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:39:44.316812 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-127f0ae0-a7f7-4c28-b69a-6874c376efef None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:39:44.318050 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-127f0ae0-a7f7-4c28-b69a-6874c376efef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:39:44.325882 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 761/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 28 15:39:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:39:44.474772 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c44c24e-c75d-4bb3-a524-8e0221831133 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:39:44.489087 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c44c24e-c75d-4bb3-a524-8e0221831133 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:39:44.494449 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681614d8-e1b5-42a3-b8c5-ba29feb4a35f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf14b64bf04b2ba74d2fe1fd41ff71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:39:44.495031 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681614d8-e1b5-42a3-b8c5-ba29feb4a35f service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:39:44.495306 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681614d8-e1b5-42a3-b8c5-ba29feb4a35f service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:39:44.495574 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681614d8-e1b5-42a3-b8c5-ba29feb4a35f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:39:44.504213 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681614d8-e1b5-42a3-b8c5-ba29feb4a35f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:39:44.505785 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681614d8-e1b5-42a3-b8c5-ba29feb4a35f service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:39:44.514894 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 766/1527] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:39:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:39:48.796999 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46179b4-9c29-41cb-8c6f-47b830332f8c None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:39:48.797547 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46179b4-9c29-41cb-8c6f-47b830332f8c None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:39:48.797547 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46179b4-9c29-41cb-8c6f-47b830332f8c None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:39:48.798903 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 762/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 15:39:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:39:48.804946 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a9697-8083-4064-a930-101279c2de57 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:39:48.805424 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a9697-8083-4064-a930-101279c2de57 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:39:48.805579 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a9697-8083-4064-a930-101279c2de57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:39:48.825353 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-911a9697-8083-4064-a930-101279c2de57 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:39:48.826586 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-911a9697-8083-4064-a930-101279c2de57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:39:48.835074 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 767/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 15:39:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:39:49.120319 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46179b4-9c29-41cb-8c6f-47b830332f8c None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:39:49.122022 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c46179b4-9c29-41cb-8c6f-47b830332f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:39:49.199478 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46179b4-9c29-41cb-8c6f-47b830332f8c None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:39:49.204836 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6bb3e6-0cdc-454e-a07c-832aeddc2fb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:39:49.205875 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6bb3e6-0cdc-454e-a07c-832aeddc2fb9 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:39:49.206335 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6bb3e6-0cdc-454e-a07c-832aeddc2fb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:39:49.206623 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6bb3e6-0cdc-454e-a07c-832aeddc2fb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:39:49.215752 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6bb3e6-0cdc-454e-a07c-832aeddc2fb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:39:49.217229 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6bb3e6-0cdc-454e-a07c-832aeddc2fb9 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:39:49.226539 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 763/1530] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:39:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:39:51.353556 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33423420-91af-4223-9e09-4d8e8cc0f1f8 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:39:51.354118 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33423420-91af-4223-9e09-4d8e8cc0f1f8 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:39:51.354118 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33423420-91af-4223-9e09-4d8e8cc0f1f8 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:39:51.355519 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 768/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 15:39:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:39:51.360981 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e48debf-5ff0-49ee-81c6-6c0ee3ad605d None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:39:51.361389 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e48debf-5ff0-49ee-81c6-6c0ee3ad605d None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:39:51.361523 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e48debf-5ff0-49ee-81c6-6c0ee3ad605d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:39:51.380786 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-7e48debf-5ff0-49ee-81c6-6c0ee3ad605d None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:39:51.381988 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7e48debf-5ff0-49ee-81c6-6c0ee3ad605d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:39:51.389586 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 764/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 15:39:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:39:51.675137 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33423420-91af-4223-9e09-4d8e8cc0f1f8 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:39:51.685645 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33423420-91af-4223-9e09-4d8e8cc0f1f8 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:39:51.689648 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb5daca-e566-4bb1-bb24-6f05d36a6d1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:39:51.690270 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5daca-e566-4bb1-bb24-6f05d36a6d1f service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:39:51.690549 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5daca-e566-4bb1-bb24-6f05d36a6d1f service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:39:51.690817 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5daca-e566-4bb1-bb24-6f05d36a6d1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:39:51.699487 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb5daca-e566-4bb1-bb24-6f05d36a6d1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:39:51.700907 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb5daca-e566-4bb1-bb24-6f05d36a6d1f service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:39:51.710622 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 769/1533] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:39:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:39:55.601459 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e59101-3e86-4df4-971f-3eb2d1c5a1cc None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:39:55.602045 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e59101-3e86-4df4-971f-3eb2d1c5a1cc None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:39:55.602045 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e59101-3e86-4df4-971f-3eb2d1c5a1cc None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:39:55.603368 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 765/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 15:39:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:39:55.609226 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4343ed-d359-4599-ad54-e6331d55f3fa None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:39:55.609663 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4343ed-d359-4599-ad54-e6331d55f3fa None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:39:55.609800 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4343ed-d359-4599-ad54-e6331d55f3fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:39:55.629014 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-9a4343ed-d359-4599-ad54-e6331d55f3fa None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:39:55.630351 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9a4343ed-d359-4599-ad54-e6331d55f3fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:39:55.637972 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 770/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 15:39:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:39:55.943742 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e59101-3e86-4df4-971f-3eb2d1c5a1cc None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:39:55.945403 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b5e59101-3e86-4df4-971f-3eb2d1c5a1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:39:56.009571 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:39:56.023274 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e59101-3e86-4df4-971f-3eb2d1c5a1cc None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:39:56.026843 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae44149-719c-451f-a088-78d29392b5e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:39:56.027441 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae44149-719c-451f-a088-78d29392b5e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:39:56.027729 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae44149-719c-451f-a088-78d29392b5e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:39:56.027981 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae44149-719c-451f-a088-78d29392b5e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:39:56.037324 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae44149-719c-451f-a088-78d29392b5e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:39:56.038768 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae44149-719c-451f-a088-78d29392b5e8 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:39:56.048137 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 766/1536] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:39:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:39:58.183818 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e95bd-7801-4542-9463-15bad6a255a1 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:39:58.184361 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e95bd-7801-4542-9463-15bad6a255a1 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:39:58.184361 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e95bd-7801-4542-9463-15bad6a255a1 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:39:58.185682 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 771/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 28 15:39:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:39:58.191919 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29288223-bc53-4f09-ba7d-3713f7b58b15 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:39:58.192376 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29288223-bc53-4f09-ba7d-3713f7b58b15 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:39:58.192471 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29288223-bc53-4f09-ba7d-3713f7b58b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:39:58.212876 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-29288223-bc53-4f09-ba7d-3713f7b58b15 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:39:58.214102 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-29288223-bc53-4f09-ba7d-3713f7b58b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:39:58.221753 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 767/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 28 15:39:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:39:58.523503 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560e95bd-7801-4542-9463-15bad6a255a1 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:39:58.525164 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-560e95bd-7801-4542-9463-15bad6a255a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:39:58.533489 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=84 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:39:58.540278 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560e95bd-7801-4542-9463-15bad6a255a1 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:39:58.543822 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c500ae1-b2c1-4cba-8a27-5b3f289a122a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:39:58.544416 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c500ae1-b2c1-4cba-8a27-5b3f289a122a service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:39:58.544696 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c500ae1-b2c1-4cba-8a27-5b3f289a122a service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:39:58.544966 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c500ae1-b2c1-4cba-8a27-5b3f289a122a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:39:58.554399 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c500ae1-b2c1-4cba-8a27-5b3f289a122a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:39:58.555807 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c500ae1-b2c1-4cba-8a27-5b3f289a122a service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:39:58.565012 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 772/1539] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:39:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:40:08.207610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=11 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:40:08.560898 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:41:21.598340 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63598f8-2b97-46c6-acef-d1ed3c203aac None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:21.598973 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63598f8-2b97-46c6-acef-d1ed3c203aac None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:21.598973 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63598f8-2b97-46c6-acef-d1ed3c203aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:21.635346 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-e63598f8-2b97-46c6-acef-d1ed3c203aac None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:21.636595 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e63598f8-2b97-46c6-acef-d1ed3c203aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:21.649418 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 768/1540] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:41:21.658929 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263f29ca-f811-4ed3-83be-24aae038508b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:21.659557 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263f29ca-f811-4ed3-83be-24aae038508b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:21.659557 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263f29ca-f811-4ed3-83be-24aae038508b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:21.686636 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-263f29ca-f811-4ed3-83be-24aae038508b None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:21.688506 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-263f29ca-f811-4ed3-83be-24aae038508b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:21.701526 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 773/1541] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:21.706210 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63598f8-2b97-46c6-acef-d1ed3c203aac None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:21.721550 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63598f8-2b97-46c6-acef-d1ed3c203aac None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:21.730140 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a5c48e-75fb-436f-a506-8730ff8bbcf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:21.730761 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5c48e-75fb-436f-a506-8730ff8bbcf8 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:21.731052 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5c48e-75fb-436f-a506-8730ff8bbcf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:21.731309 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5c48e-75fb-436f-a506-8730ff8bbcf8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:21.731994 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a5c48e-75fb-436f-a506-8730ff8bbcf8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:41:21.732295 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a5c48e-75fb-436f-a506-8730ff8bbcf8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:21.734663 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a5c48e-75fb-436f-a506-8730ff8bbcf8 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:21.743098 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b719a-0602-4bcb-a844-52384c2fbe33 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:21.743403 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b719a-0602-4bcb-a844-52384c2fbe33 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:21.743646 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b719a-0602-4bcb-a844-52384c2fbe33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:21.753374 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 769/1542] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:41:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:41:21.759651 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4376fe-dbda-42c7-abac-e894c21adb18 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:21.759950 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4376fe-dbda-42c7-abac-e894c21adb18 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:21.760201 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4376fe-dbda-42c7-abac-e894c21adb18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:21.773904 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-fe8b719a-0602-4bcb-a844-52384c2fbe33 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:21.775109 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-fe8b719a-0602-4bcb-a844-52384c2fbe33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:21.785914 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 774/1543] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:41:21.787502 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7aef22-7190-41ff-967e-4d8beb14d43c None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:21.787985 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7aef22-7190-41ff-967e-4d8beb14d43c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:21.788400 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7aef22-7190-41ff-967e-4d8beb14d43c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:21.794977 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-fc4376fe-dbda-42c7-abac-e894c21adb18 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:21.796585 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-fc4376fe-dbda-42c7-abac-e894c21adb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:21.814101 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-1e7aef22-7190-41ff-967e-4d8beb14d43c None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:21.815331 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1e7aef22-7190-41ff-967e-4d8beb14d43c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:21.817240 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 770/1544] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:21.818370 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4376fe-dbda-42c7-abac-e894c21adb18 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:21.826167 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 775/1545] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:21.830924 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913d2ad-92c6-40a7-a6fd-dc7556e46846 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:21.830924 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913d2ad-92c6-40a7-a6fd-dc7556e46846 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:21.830924 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913d2ad-92c6-40a7-a6fd-dc7556e46846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:21.839463 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4376fe-dbda-42c7-abac-e894c21adb18 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:21.852809 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e636dac6-0764-476c-a3ab-35e4e1f32e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:21.854420 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e636dac6-0764-476c-a3ab-35e4e1f32e3e admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:21.855084 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e636dac6-0764-476c-a3ab-35e4e1f32e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:21.855614 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e636dac6-0764-476c-a3ab-35e4e1f32e3e admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:21.856461 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e636dac6-0764-476c-a3ab-35e4e1f32e3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:21.857706 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: WARNING py.warnings [None req-e636dac6-0764-476c-a3ab-35e4e1f32e3e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 15:41:21.857706 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: warnings.warn(msg) Sep 28 15:41:21.857706 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: Sep 28 15:41:21.859520 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e636dac6-0764-476c-a3ab-35e4e1f32e3e admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:21.869883 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-a913d2ad-92c6-40a7-a6fd-dc7556e46846 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:21.869883 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a913d2ad-92c6-40a7-a6fd-dc7556e46846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:21.877347 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 776/1546] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:21.881933 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659049a-a9a8-4100-a05f-dcead1aebd60 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:21.881933 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659049a-a9a8-4100-a05f-dcead1aebd60 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:21.881933 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659049a-a9a8-4100-a05f-dcead1aebd60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:21.919161 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-6659049a-a9a8-4100-a05f-dcead1aebd60 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:21.920412 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6659049a-a9a8-4100-a05f-dcead1aebd60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:21.932406 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 777/1547] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:21.934118 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8996e614-a94c-4253-89cc-51bbbb476d36 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:21.934405 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8996e614-a94c-4253-89cc-51bbbb476d36 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:21.934719 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8996e614-a94c-4253-89cc-51bbbb476d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:21.940338 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 771/1548] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:41:21 2022] POST /identity/v3/projects => generated 379 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:41:21.941571 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e636dac6-0764-476c-a3ab-35e4e1f32e3e admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:21.957292 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-8996e614-a94c-4253-89cc-51bbbb476d36 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:21.958683 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8996e614-a94c-4253-89cc-51bbbb476d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:22.015130 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e636dac6-0764-476c-a3ab-35e4e1f32e3e admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:22.019803 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a2884b-00dd-4828-8d0c-042e120ebf14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:22.020610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2884b-00dd-4828-8d0c-042e120ebf14 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:22.020825 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2884b-00dd-4828-8d0c-042e120ebf14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:22.021202 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a2884b-00dd-4828-8d0c-042e120ebf14 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:22.021837 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a2884b-00dd-4828-8d0c-042e120ebf14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:41:22.022349 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a2884b-00dd-4828-8d0c-042e120ebf14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:22.022566 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 778/1549] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:22.023797 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8d3cea-1caa-49f7-8284-67a75f89caf9 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:22.024062 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a2884b-00dd-4828-8d0c-042e120ebf14 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:22.024176 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8d3cea-1caa-49f7-8284-67a75f89caf9 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:22.024437 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8d3cea-1caa-49f7-8284-67a75f89caf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:22.041883 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 772/1550] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:41:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:41:22.041883 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00a03b-7460-457f-a08d-0b2ade3562bc None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:22.041883 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00a03b-7460-457f-a08d-0b2ade3562bc None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:22.041883 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00a03b-7460-457f-a08d-0b2ade3562bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:22.048899 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-8a8d3cea-1caa-49f7-8284-67a75f89caf9 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:22.050175 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8a8d3cea-1caa-49f7-8284-67a75f89caf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:22.058291 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 779/1551] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:22.059089 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8d3cea-1caa-49f7-8284-67a75f89caf9 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:22.060806 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-7a00a03b-7460-457f-a08d-0b2ade3562bc None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:22.062013 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7a00a03b-7460-457f-a08d-0b2ade3562bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:22.069211 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8d3cea-1caa-49f7-8284-67a75f89caf9 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:22.070191 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 773/1552] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:22.070963 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a00a03b-7460-457f-a08d-0b2ade3562bc None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:22.074157 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d26050-55cf-4e0d-ae0f-1b50493e2e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:22.074746 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d26050-55cf-4e0d-ae0f-1b50493e2e03 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:22.075028 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d26050-55cf-4e0d-ae0f-1b50493e2e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:22.075312 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d26050-55cf-4e0d-ae0f-1b50493e2e03 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:22.076650 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d26050-55cf-4e0d-ae0f-1b50493e2e03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:41:22.076986 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d26050-55cf-4e0d-ae0f-1b50493e2e03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:22.078452 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: WARNING py.warnings [None req-c2d26050-55cf-4e0d-ae0f-1b50493e2e03 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 28 15:41:22.078452 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: warnings.warn(msg) Sep 28 15:41:22.078452 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: Sep 28 15:41:22.079362 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d26050-55cf-4e0d-ae0f-1b50493e2e03 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:22.083000 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a00a03b-7460-457f-a08d-0b2ade3562bc None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:22.087653 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f58b5-088e-4019-bce8-e360aa37d2d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:22.088266 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f58b5-088e-4019-bce8-e360aa37d2d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:22.088640 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f58b5-088e-4019-bce8-e360aa37d2d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:22.088856 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f58b5-088e-4019-bce8-e360aa37d2d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:22.089484 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3f58b5-088e-4019-bce8-e360aa37d2d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:22.091452 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3f58b5-088e-4019-bce8-e360aa37d2d8 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:22.092867 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 780/1553] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:41:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:41:22.094142 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bff822-ddef-4f6d-8801-47fd4351e5d9 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:22.094426 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bff822-ddef-4f6d-8801-47fd4351e5d9 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:22.094696 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bff822-ddef-4f6d-8801-47fd4351e5d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:22.115158 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-24bff822-ddef-4f6d-8801-47fd4351e5d9 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:22.115654 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 774/1554] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:41:22 2022] POST /identity/v3/users => generated 290 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:41:22.116338 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-24bff822-ddef-4f6d-8801-47fd4351e5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:22.116741 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c00d6-5d03-4fa6-b521-9f8220054b88 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:22.117024 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c00d6-5d03-4fa6-b521-9f8220054b88 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:22.117301 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12c00d6-5d03-4fa6-b521-9f8220054b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:22.124935 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 781/1555] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:22.126538 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bff822-ddef-4f6d-8801-47fd4351e5d9 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:22.135898 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bff822-ddef-4f6d-8801-47fd4351e5d9 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:22.141036 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2b879c-64dc-4fec-a92d-4ba00d960431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:22.141655 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2b879c-64dc-4fec-a92d-4ba00d960431 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:22.141945 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2b879c-64dc-4fec-a92d-4ba00d960431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:22.142217 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2b879c-64dc-4fec-a92d-4ba00d960431 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:22.142848 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2b879c-64dc-4fec-a92d-4ba00d960431 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:41:22.143146 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2b879c-64dc-4fec-a92d-4ba00d960431 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:22.144730 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2b879c-64dc-4fec-a92d-4ba00d960431 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:22.148174 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-b12c00d6-5d03-4fa6-b521-9f8220054b88 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:22.149496 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b12c00d6-5d03-4fa6-b521-9f8220054b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:22.156298 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 782/1556] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:41:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:41:22.157063 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2b879c-64dc-4fec-a92d-4ba00d960431 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:22.162569 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 775/1557] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:22.164529 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f30a9-bd26-4b47-996b-64f47ae71c6f None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:22.164943 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f30a9-bd26-4b47-996b-64f47ae71c6f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:22.165219 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f30a9-bd26-4b47-996b-64f47ae71c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:22.172689 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2b879c-64dc-4fec-a92d-4ba00d960431 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:22.177181 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308abf48-96ce-4953-9e21-32e36d52b29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:22.177771 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308abf48-96ce-4953-9e21-32e36d52b29a admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:22.178075 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308abf48-96ce-4953-9e21-32e36d52b29a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:22.178346 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308abf48-96ce-4953-9e21-32e36d52b29a admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:22.178961 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308abf48-96ce-4953-9e21-32e36d52b29a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:22.180855 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308abf48-96ce-4953-9e21-32e36d52b29a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:22.184646 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-181f30a9-bd26-4b47-996b-64f47ae71c6f None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:22.186404 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-181f30a9-bd26-4b47-996b-64f47ae71c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:22.194280 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 776/1558] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:22.199201 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8291d0-2fa9-4719-97e5-fa30b47883ec None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:22.199479 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8291d0-2fa9-4719-97e5-fa30b47883ec None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:22.199751 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8291d0-2fa9-4719-97e5-fa30b47883ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:22.209494 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 783/1559] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:41:22 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:41:22.210328 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308abf48-96ce-4953-9e21-32e36d52b29a admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:22.223140 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-be8291d0-2fa9-4719-97e5-fa30b47883ec None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:22.224330 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-be8291d0-2fa9-4719-97e5-fa30b47883ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:22.272979 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308abf48-96ce-4953-9e21-32e36d52b29a admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:22.279388 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92746fd1-778a-4e42-861c-5430ffb292d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:22.279999 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92746fd1-778a-4e42-861c-5430ffb292d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:22.280278 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92746fd1-778a-4e42-861c-5430ffb292d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:22.280551 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92746fd1-778a-4e42-861c-5430ffb292d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:22.281200 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92746fd1-778a-4e42-861c-5430ffb292d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:22.281343 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:41:22.283163 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92746fd1-778a-4e42-861c-5430ffb292d9 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:22.291714 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 777/1560] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:22.293158 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8291d0-2fa9-4719-97e5-fa30b47883ec None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:22.296265 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:41:22.306816 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8291d0-2fa9-4719-97e5-fa30b47883ec None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:22.312292 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cef9be-4e21-4f4d-8a99-34535f8a606d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:22.312906 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cef9be-4e21-4f4d-8a99-34535f8a606d admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:22.313183 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cef9be-4e21-4f4d-8a99-34535f8a606d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:22.313457 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cef9be-4e21-4f4d-8a99-34535f8a606d admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:22.314146 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cef9be-4e21-4f4d-8a99-34535f8a606d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:22.314982 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 784/1561] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:41:22 2022] POST /identity/v3/projects => generated 375 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:41:22.315759 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cef9be-4e21-4f4d-8a99-34535f8a606d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:22.316212 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327adf4f-a8e1-4aa7-af1b-8ff8240f05dc None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:22.316499 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327adf4f-a8e1-4aa7-af1b-8ff8240f05dc None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:22.316777 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327adf4f-a8e1-4aa7-af1b-8ff8240f05dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:22.328651 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 778/1562] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:41:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:41:22.329402 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cef9be-4e21-4f4d-8a99-34535f8a606d admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:22.338713 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-327adf4f-a8e1-4aa7-af1b-8ff8240f05dc None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:22.338967 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cef9be-4e21-4f4d-8a99-34535f8a606d admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:22.339915 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-327adf4f-a8e1-4aa7-af1b-8ff8240f05dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:22.398875 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f63d6-1b2b-46ef-a07d-ade6817b6eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:22.399489 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f63d6-1b2b-46ef-a07d-ade6817b6eec admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:22.399762 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f63d6-1b2b-46ef-a07d-ade6817b6eec admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:22.400028 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f63d6-1b2b-46ef-a07d-ade6817b6eec admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:22.400501 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 785/1563] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:22.400690 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5f63d6-1b2b-46ef-a07d-ade6817b6eec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:22.401818 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8a14a-5e37-4d7e-8d8f-6c18f3efc07a None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:22.402137 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8a14a-5e37-4d7e-8d8f-6c18f3efc07a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:22.402408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8a14a-5e37-4d7e-8d8f-6c18f3efc07a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:22.402743 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5f63d6-1b2b-46ef-a07d-ade6817b6eec admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:22.422349 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-fee8a14a-5e37-4d7e-8d8f-6c18f3efc07a None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:22.423599 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-fee8a14a-5e37-4d7e-8d8f-6c18f3efc07a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:22.430692 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 779/1564] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:41:22 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:41:22.431398 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f63d6-1b2b-46ef-a07d-ade6817b6eec admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:22.436219 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 786/1565] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:22.436963 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee8a14a-5e37-4d7e-8d8f-6c18f3efc07a None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:22.446276 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee8a14a-5e37-4d7e-8d8f-6c18f3efc07a None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:22.495853 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f63d6-1b2b-46ef-a07d-ade6817b6eec admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:22.501423 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf3f3d5-b9ba-4c44-abbf-52596bb9d259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:22.502062 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf3f3d5-b9ba-4c44-abbf-52596bb9d259 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:22.502338 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf3f3d5-b9ba-4c44-abbf-52596bb9d259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:22.502452 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b8f91b-9db0-4a69-be23-8be27cbce9d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:22.502632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf3f3d5-b9ba-4c44-abbf-52596bb9d259 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:22.503044 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8f91b-9db0-4a69-be23-8be27cbce9d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:22.503267 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf3f3d5-b9ba-4c44-abbf-52596bb9d259 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:41:22.503377 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8f91b-9db0-4a69-be23-8be27cbce9d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:22.503536 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf3f3d5-b9ba-4c44-abbf-52596bb9d259 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:22.503661 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8f91b-9db0-4a69-be23-8be27cbce9d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:22.504202 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b8f91b-9db0-4a69-be23-8be27cbce9d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:22.505117 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf3f3d5-b9ba-4c44-abbf-52596bb9d259 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:22.505233 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: WARNING py.warnings [None req-02b8f91b-9db0-4a69-be23-8be27cbce9d9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 15:41:22.505233 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: warnings.warn(msg) Sep 28 15:41:22.505233 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: Sep 28 15:41:22.506422 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b8f91b-9db0-4a69-be23-8be27cbce9d9 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:22.516589 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 780/1566] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:41:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:41:22.517543 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf3f3d5-b9ba-4c44-abbf-52596bb9d259 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:22.527218 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf3f3d5-b9ba-4c44-abbf-52596bb9d259 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:22.531998 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc19ed20-fccb-4961-b6e6-24d72f6aa569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:22.532575 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19ed20-fccb-4961-b6e6-24d72f6aa569 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:22.532851 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19ed20-fccb-4961-b6e6-24d72f6aa569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:22.533117 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc19ed20-fccb-4961-b6e6-24d72f6aa569 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:22.533786 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc19ed20-fccb-4961-b6e6-24d72f6aa569 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:41:22.534108 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc19ed20-fccb-4961-b6e6-24d72f6aa569 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:22.534622 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 787/1567] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:41:22 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:41:22.535415 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b8f91b-9db0-4a69-be23-8be27cbce9d9 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:22.535760 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc19ed20-fccb-4961-b6e6-24d72f6aa569 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:22.545614 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b8f91b-9db0-4a69-be23-8be27cbce9d9 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:22.550238 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba2b410-11bf-48bc-a2aa-f3f0098b2ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:22.550688 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 781/1568] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:41:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:41:22.551095 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2b410-11bf-48bc-a2aa-f3f0098b2ed2 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:22.551443 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2b410-11bf-48bc-a2aa-f3f0098b2ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:22.551551 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc19ed20-fccb-4961-b6e6-24d72f6aa569 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:22.551710 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2b410-11bf-48bc-a2aa-f3f0098b2ed2 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:22.552323 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba2b410-11bf-48bc-a2aa-f3f0098b2ed2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:22.554243 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba2b410-11bf-48bc-a2aa-f3f0098b2ed2 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:22.567280 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc19ed20-fccb-4961-b6e6-24d72f6aa569 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:22.572630 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e28babf-eaa5-4885-bc6b-7689cc01b3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:22.573238 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28babf-eaa5-4885-bc6b-7689cc01b3eb admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:22.573501 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28babf-eaa5-4885-bc6b-7689cc01b3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:22.573773 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e28babf-eaa5-4885-bc6b-7689cc01b3eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:22.574491 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e28babf-eaa5-4885-bc6b-7689cc01b3eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:22.576062 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e28babf-eaa5-4885-bc6b-7689cc01b3eb admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:22.579823 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 788/1569] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:41:22 2022] POST /identity/v3/users => generated 288 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:41:22.579823 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba2b410-11bf-48bc-a2aa-f3f0098b2ed2 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:22.586915 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba2b410-11bf-48bc-a2aa-f3f0098b2ed2 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:22.589103 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 782/1570] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:41:22 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:41:22.589917 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e28babf-eaa5-4885-bc6b-7689cc01b3eb admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:22.594641 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be59ff98-b8f8-4c85-92c4-3e015a2dd87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:22.595231 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59ff98-b8f8-4c85-92c4-3e015a2dd87d admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:22.595510 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59ff98-b8f8-4c85-92c4-3e015a2dd87d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:22.595774 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59ff98-b8f8-4c85-92c4-3e015a2dd87d admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:22.596397 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be59ff98-b8f8-4c85-92c4-3e015a2dd87d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:41:22.596696 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be59ff98-b8f8-4c85-92c4-3e015a2dd87d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:22.598307 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be59ff98-b8f8-4c85-92c4-3e015a2dd87d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:22.599128 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e28babf-eaa5-4885-bc6b-7689cc01b3eb admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:22.603749 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c8d39-c54c-46b6-bac6-c067cdcfcf52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:22.604319 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c8d39-c54c-46b6-bac6-c067cdcfcf52 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:22.604595 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c8d39-c54c-46b6-bac6-c067cdcfcf52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:22.604863 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c8d39-c54c-46b6-bac6-c067cdcfcf52 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:22.605481 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9c8d39-c54c-46b6-bac6-c067cdcfcf52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:22.608003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9c8d39-c54c-46b6-bac6-c067cdcfcf52 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:22.618147 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 789/1571] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:41:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:41:22.618147 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be59ff98-b8f8-4c85-92c4-3e015a2dd87d admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:22.624931 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be59ff98-b8f8-4c85-92c4-3e015a2dd87d admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:22.630182 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031c6727-9dd2-4758-bb2b-aedb2ecd4114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:22.630893 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 783/1572] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:41:22 2022] POST /identity/v3/users => generated 290 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:41:22.631036 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031c6727-9dd2-4758-bb2b-aedb2ecd4114 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:22.631229 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031c6727-9dd2-4758-bb2b-aedb2ecd4114 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:22.631546 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c8d39-c54c-46b6-bac6-c067cdcfcf52 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:22.631683 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031c6727-9dd2-4758-bb2b-aedb2ecd4114 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:22.632300 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031c6727-9dd2-4758-bb2b-aedb2ecd4114 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:41:22.632608 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031c6727-9dd2-4758-bb2b-aedb2ecd4114 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:22.634313 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031c6727-9dd2-4758-bb2b-aedb2ecd4114 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:22.643115 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9c8d39-c54c-46b6-bac6-c067cdcfcf52 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:22.654297 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 790/1573] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:41:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:41:22.654487 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b5153e-1513-41e9-8b0f-17bed9951514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:22.654617 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031c6727-9dd2-4758-bb2b-aedb2ecd4114 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:22.654706 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5153e-1513-41e9-8b0f-17bed9951514 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:22.654706 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5153e-1513-41e9-8b0f-17bed9951514 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:22.654706 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5153e-1513-41e9-8b0f-17bed9951514 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:22.654706 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b5153e-1513-41e9-8b0f-17bed9951514 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:22.654706 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b5153e-1513-41e9-8b0f-17bed9951514 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:22.660496 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031c6727-9dd2-4758-bb2b-aedb2ecd4114 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:22.666114 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0218a289-0e0a-4c6f-ab6b-d806dc026462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:22.666714 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0218a289-0e0a-4c6f-ab6b-d806dc026462 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:22.667111 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0218a289-0e0a-4c6f-ab6b-d806dc026462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:22.667391 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0218a289-0e0a-4c6f-ab6b-d806dc026462 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:22.668078 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0218a289-0e0a-4c6f-ab6b-d806dc026462 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:22.670970 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0218a289-0e0a-4c6f-ab6b-d806dc026462 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:22.682363 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 784/1574] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:41:22 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:41:22.682535 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 791/1575] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:41:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:41:22.682998 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b5153e-1513-41e9-8b0f-17bed9951514 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:22.683217 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0218a289-0e0a-4c6f-ab6b-d806dc026462 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:22.691713 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:41:22.694741 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0218a289-0e0a-4c6f-ab6b-d806dc026462 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:22.711848 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:41:22.751990 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b5153e-1513-41e9-8b0f-17bed9951514 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:22.755389 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91b9f18-f8da-422e-9b19-5295ecc35c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:22.755989 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b9f18-f8da-422e-9b19-5295ecc35c7a admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:22.756279 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b9f18-f8da-422e-9b19-5295ecc35c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:22.756550 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b9f18-f8da-422e-9b19-5295ecc35c7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:22.756971 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5911d6d3-19f8-4d6c-b09d-4d2b3d0201a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:22.757285 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91b9f18-f8da-422e-9b19-5295ecc35c7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:22.757561 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911d6d3-19f8-4d6c-b09d-4d2b3d0201a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:22.757831 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911d6d3-19f8-4d6c-b09d-4d2b3d0201a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:22.758122 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911d6d3-19f8-4d6c-b09d-4d2b3d0201a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:22.758739 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5911d6d3-19f8-4d6c-b09d-4d2b3d0201a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:22.759029 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91b9f18-f8da-422e-9b19-5295ecc35c7a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:22.760605 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5911d6d3-19f8-4d6c-b09d-4d2b3d0201a0 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:22.768935 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 792/1576] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:41:22 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:41:22.769882 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91b9f18-f8da-422e-9b19-5295ecc35c7a admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:22.780417 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91b9f18-f8da-422e-9b19-5295ecc35c7a admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:22.786230 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fe9799-a473-42a7-8286-7fd31d961f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:22.786873 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe9799-a473-42a7-8286-7fd31d961f48 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:22.787177 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe9799-a473-42a7-8286-7fd31d961f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:22.787729 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe9799-a473-42a7-8286-7fd31d961f48 admin admin] PATH_INFO: `/v3/projects/e4c3c0db462e41488735ffe3589beb8a/users/50450686eb4d45e5828d186c6ea8a515/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:22.788637 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 785/1577] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:41:22 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:41:22.789432 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5911d6d3-19f8-4d6c-b09d-4d2b3d0201a0 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:22.805459 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fe9799-a473-42a7-8286-7fd31d961f48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4c3c0db462e41488735ffe3589beb8a, user_id=50450686eb4d45e5828d186c6ea8a515, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:22.808155 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fe9799-a473-42a7-8286-7fd31d961f48 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:22.821541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 793/1578] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:41:22 2022] PUT /identity/v3/projects/e4c3c0db462e41488735ffe3589beb8a/users/50450686eb4d45e5828d186c6ea8a515/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:41:22.832805 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fe9799-a473-42a7-8286-7fd31d961f48 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:22.866634 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5911d6d3-19f8-4d6c-b09d-4d2b3d0201a0 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:22.887768 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fe9799-a473-42a7-8286-7fd31d961f48 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:22.894601 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff866c9-6623-45ab-b563-35a0a4636cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:22.895226 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff866c9-6623-45ab-b563-35a0a4636cb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:22.895513 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff866c9-6623-45ab-b563-35a0a4636cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:22.895837 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff866c9-6623-45ab-b563-35a0a4636cb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:22.896513 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff866c9-6623-45ab-b563-35a0a4636cb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:22.898315 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff866c9-6623-45ab-b563-35a0a4636cb2 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:22.908929 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 794/1579] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:41:22 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:41:22.909706 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff866c9-6623-45ab-b563-35a0a4636cb2 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:22.921547 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff866c9-6623-45ab-b563-35a0a4636cb2 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:22.926339 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c14cb9c-9a3f-4378-b6de-7e15748b795f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:22.926915 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14cb9c-9a3f-4378-b6de-7e15748b795f admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:22.927194 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14cb9c-9a3f-4378-b6de-7e15748b795f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:22.927465 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14cb9c-9a3f-4378-b6de-7e15748b795f admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:22.928066 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c14cb9c-9a3f-4378-b6de-7e15748b795f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:22.928346 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d9142-b25e-4642-9157-09aaa0669f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:22.928975 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d9142-b25e-4642-9157-09aaa0669f0b admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:22.929250 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d9142-b25e-4642-9157-09aaa0669f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:22.929517 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d9142-b25e-4642-9157-09aaa0669f0b admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:22.929996 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c14cb9c-9a3f-4378-b6de-7e15748b795f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:22.930189 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15d9142-b25e-4642-9157-09aaa0669f0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:22.932100 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15d9142-b25e-4642-9157-09aaa0669f0b admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:22.957112 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 795/1580] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:41:22 2022] POST /identity/v3/projects => generated 389 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:41:22.960718 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 786/1581] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:41:22 2022] POST /identity/v3/projects => generated 391 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:41:22.961062 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d9142-b25e-4642-9157-09aaa0669f0b admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:22.961189 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c14cb9c-9a3f-4378-b6de-7e15748b795f admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:22.986251 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:41:23.022797 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c14cb9c-9a3f-4378-b6de-7e15748b795f admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:23.025486 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d9142-b25e-4642-9157-09aaa0669f0b admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:23.027455 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392c7876-d3d0-4406-80f3-87eb108903ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:23.028015 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c7876-d3d0-4406-80f3-87eb108903ee admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:23.028222 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c7876-d3d0-4406-80f3-87eb108903ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:23.028476 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c7876-d3d0-4406-80f3-87eb108903ee admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:23.029617 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392c7876-d3d0-4406-80f3-87eb108903ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:23.031099 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809a83f8-6444-4215-a7e4-5eed430a8764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:23.031565 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392c7876-d3d0-4406-80f3-87eb108903ee admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:23.031691 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a83f8-6444-4215-a7e4-5eed430a8764 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:23.031991 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a83f8-6444-4215-a7e4-5eed430a8764 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:23.032280 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a83f8-6444-4215-a7e4-5eed430a8764 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:23.032959 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809a83f8-6444-4215-a7e4-5eed430a8764 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:23.034604 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809a83f8-6444-4215-a7e4-5eed430a8764 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:23.042406 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:41:23.045329 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 787/1582] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:41:22 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:41:23.046283 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809a83f8-6444-4215-a7e4-5eed430a8764 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:23.056153 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809a83f8-6444-4215-a7e4-5eed430a8764 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:23.059279 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 796/1583] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:41:22 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:41:23.059279 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392c7876-d3d0-4406-80f3-87eb108903ee admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:23.060880 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ebf6c2-1e97-4931-a962-eeb4b667a27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:23.061466 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ebf6c2-1e97-4931-a962-eeb4b667a27b admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:23.061745 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ebf6c2-1e97-4931-a962-eeb4b667a27b admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:23.062062 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ebf6c2-1e97-4931-a962-eeb4b667a27b admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:23.062680 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ebf6c2-1e97-4931-a962-eeb4b667a27b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:23.064294 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ebf6c2-1e97-4931-a962-eeb4b667a27b admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:23.066164 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392c7876-d3d0-4406-80f3-87eb108903ee admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:23.070662 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337a3b8e-de6b-4d6d-bdd6-f0b6bc6dda38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:23.071243 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a3b8e-de6b-4d6d-bdd6-f0b6bc6dda38 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:23.071518 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a3b8e-de6b-4d6d-bdd6-f0b6bc6dda38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:23.071784 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a3b8e-de6b-4d6d-bdd6-f0b6bc6dda38 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:23.072382 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337a3b8e-de6b-4d6d-bdd6-f0b6bc6dda38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:23.074361 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 788/1584] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:41:23 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:41:23.074491 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337a3b8e-de6b-4d6d-bdd6-f0b6bc6dda38 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:23.075519 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ed786-e448-4c5c-b1b8-9b61375dd0f4 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:23.075801 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ed786-e448-4c5c-b1b8-9b61375dd0f4 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:23.076065 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ed786-e448-4c5c-b1b8-9b61375dd0f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:23.097640 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 797/1585] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:41:23 2022] POST /identity/v3/users => generated 287 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:41:23.098794 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337a3b8e-de6b-4d6d-bdd6-f0b6bc6dda38 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:23.108439 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-4b1ed786-e448-4c5c-b1b8-9b61375dd0f4 None None] MFA Rules not processed for user `50450686eb4d45e5828d186c6ea8a515`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:23.109678 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4b1ed786-e448-4c5c-b1b8-9b61375dd0f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:23.110617 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337a3b8e-de6b-4d6d-bdd6-f0b6bc6dda38 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:23.115209 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c6348-42df-4c86-916b-500255e56ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:23.115828 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c6348-42df-4c86-916b-500255e56ffe admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:23.116076 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c6348-42df-4c86-916b-500255e56ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:23.116352 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c6348-42df-4c86-916b-500255e56ffe admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:23.116982 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350c6348-42df-4c86-916b-500255e56ffe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:23.118603 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350c6348-42df-4c86-916b-500255e56ffe admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:23.129344 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 798/1586] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:41:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:41:23.130755 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c6348-42df-4c86-916b-500255e56ffe admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:23.141761 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c6348-42df-4c86-916b-500255e56ffe admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:23.146320 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc68d1c-bb7d-4dc0-985f-8d2668902cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:23.146898 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc68d1c-bb7d-4dc0-985f-8d2668902cf4 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:23.147175 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc68d1c-bb7d-4dc0-985f-8d2668902cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:23.147444 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc68d1c-bb7d-4dc0-985f-8d2668902cf4 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:23.148047 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc68d1c-bb7d-4dc0-985f-8d2668902cf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:23.149967 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc68d1c-bb7d-4dc0-985f-8d2668902cf4 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:23.172803 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 799/1587] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:41:23 2022] POST /identity/v3/users => generated 295 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:41:23.173590 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc68d1c-bb7d-4dc0-985f-8d2668902cf4 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:23.177722 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 789/1588] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:23.177722 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1ed786-e448-4c5c-b1b8-9b61375dd0f4 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:23.182905 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc68d1c-bb7d-4dc0-985f-8d2668902cf4 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:23.187453 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248bb78d-dd56-487f-8dcb-d1389d9d39c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:23.188039 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248bb78d-dd56-487f-8dcb-d1389d9d39c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:23.188184 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1ed786-e448-4c5c-b1b8-9b61375dd0f4 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:23.188309 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248bb78d-dd56-487f-8dcb-d1389d9d39c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:23.188578 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248bb78d-dd56-487f-8dcb-d1389d9d39c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:23.189202 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248bb78d-dd56-487f-8dcb-d1389d9d39c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:23.191162 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248bb78d-dd56-487f-8dcb-d1389d9d39c8 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:23.193288 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba28a261-0092-4be5-9a62-4c20d5a21d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:23.193971 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28a261-0092-4be5-9a62-4c20d5a21d1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:23.194278 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28a261-0092-4be5-9a62-4c20d5a21d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:23.194544 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28a261-0092-4be5-9a62-4c20d5a21d1d admin admin] PATH_INFO: `/v3/projects/62692d2b160040879aecaf4d44ad19b3/users/634c0af05b334b488f2fa358e574172b/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:23.210485 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba28a261-0092-4be5-9a62-4c20d5a21d1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=62692d2b160040879aecaf4d44ad19b3, user_id=634c0af05b334b488f2fa358e574172b, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:23.211505 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: WARNING py.warnings [None req-ba28a261-0092-4be5-9a62-4c20d5a21d1d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 15:41:23.211505 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: warnings.warn(msg) Sep 28 15:41:23.211505 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: Sep 28 15:41:23.213289 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 800/1589] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:41:23 2022] POST /identity/v3/users => generated 296 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:41:23.213429 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba28a261-0092-4be5-9a62-4c20d5a21d1d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:23.214122 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248bb78d-dd56-487f-8dcb-d1389d9d39c8 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:23.225217 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248bb78d-dd56-487f-8dcb-d1389d9d39c8 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:23.228436 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 790/1590] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:41:23 2022] PUT /identity/v3/projects/62692d2b160040879aecaf4d44ad19b3/users/634c0af05b334b488f2fa358e574172b/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:41:23.232093 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba28a261-0092-4be5-9a62-4c20d5a21d1d admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:23.286848 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba81f7e-1eeb-4a97-8c68-fac1dcb529ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:23.287458 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba81f7e-1eeb-4a97-8c68-fac1dcb529ff admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:23.287732 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba81f7e-1eeb-4a97-8c68-fac1dcb529ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:23.288000 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba81f7e-1eeb-4a97-8c68-fac1dcb529ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:23.288631 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba81f7e-1eeb-4a97-8c68-fac1dcb529ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:23.293699 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba81f7e-1eeb-4a97-8c68-fac1dcb529ff admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:23.298127 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba28a261-0092-4be5-9a62-4c20d5a21d1d admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:23.300295 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 801/1591] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:41:23 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:41:23.301292 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba81f7e-1eeb-4a97-8c68-fac1dcb529ff admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:23.303581 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1b0da-7b90-44a8-8d90-a044a00bdb97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:23.304241 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1b0da-7b90-44a8-8d90-a044a00bdb97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:23.304515 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1b0da-7b90-44a8-8d90-a044a00bdb97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:23.304788 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1b0da-7b90-44a8-8d90-a044a00bdb97 admin admin] PATH_INFO: `/v3/projects/2af27dac96724c4592f66af8801ff80b/users/97b222b27b6845e3848db6666db92076/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:23.311243 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba81f7e-1eeb-4a97-8c68-fac1dcb529ff admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:23.315778 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f6f626-6a8b-4c00-b787-c3cf3329b49b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:23.316364 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6f626-6a8b-4c00-b787-c3cf3329b49b admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:23.316650 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6f626-6a8b-4c00-b787-c3cf3329b49b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:23.316924 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6f626-6a8b-4c00-b787-c3cf3329b49b admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:23.317563 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f6f626-6a8b-4c00-b787-c3cf3329b49b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:23.319199 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f6f626-6a8b-4c00-b787-c3cf3329b49b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:23.319460 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb1b0da-7b90-44a8-8d90-a044a00bdb97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2af27dac96724c4592f66af8801ff80b, user_id=97b222b27b6845e3848db6666db92076, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:23.322114 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb1b0da-7b90-44a8-8d90-a044a00bdb97 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:23.326095 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:41:23.331005 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 802/1592] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:41:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:41:23.335232 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f6f626-6a8b-4c00-b787-c3cf3329b49b admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:23.338468 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 791/1593] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:41:23 2022] PUT /identity/v3/projects/2af27dac96724c4592f66af8801ff80b/users/97b222b27b6845e3848db6666db92076/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:41:23.339259 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1b0da-7b90-44a8-8d90-a044a00bdb97 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:23.345783 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f6f626-6a8b-4c00-b787-c3cf3329b49b admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:23.400873 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1b0da-7b90-44a8-8d90-a044a00bdb97 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:23.402089 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486121f2-e7a9-40b6-b55a-2ecc53604b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:23.402746 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486121f2-e7a9-40b6-b55a-2ecc53604b9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:23.403027 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486121f2-e7a9-40b6-b55a-2ecc53604b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:23.403295 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486121f2-e7a9-40b6-b55a-2ecc53604b9c admin admin] PATH_INFO: `/v3/projects/6822e0b3eedd404895cd4c0c64082344/users/922e0b1b4717422abb6f4e93ea15a86d/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:23.406090 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d74672-1661-482e-94f0-211c9c1e4238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:23.406685 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d74672-1661-482e-94f0-211c9c1e4238 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:23.406959 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d74672-1661-482e-94f0-211c9c1e4238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:23.407222 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d74672-1661-482e-94f0-211c9c1e4238 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:23.407850 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d74672-1661-482e-94f0-211c9c1e4238 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:23.409428 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d74672-1661-482e-94f0-211c9c1e4238 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:23.418971 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486121f2-e7a9-40b6-b55a-2ecc53604b9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6822e0b3eedd404895cd4c0c64082344, user_id=922e0b1b4717422abb6f4e93ea15a86d, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:23.419602 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 792/1594] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:41:23 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:41:23.421648 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486121f2-e7a9-40b6-b55a-2ecc53604b9c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:23.423341 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d74672-1661-482e-94f0-211c9c1e4238 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:23.430174 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:41:23.433162 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d74672-1661-482e-94f0-211c9c1e4238 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:23.436133 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 803/1595] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:41:23 2022] PUT /identity/v3/projects/6822e0b3eedd404895cd4c0c64082344/users/922e0b1b4717422abb6f4e93ea15a86d/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:41:23.436826 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486121f2-e7a9-40b6-b55a-2ecc53604b9c admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:23.489948 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d09209f-a525-4d6c-9be5-eb970bbf517e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:23.490550 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d09209f-a525-4d6c-9be5-eb970bbf517e admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:23.490822 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d09209f-a525-4d6c-9be5-eb970bbf517e admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:23.491088 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d09209f-a525-4d6c-9be5-eb970bbf517e admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:23.491699 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d09209f-a525-4d6c-9be5-eb970bbf517e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:23.493571 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d09209f-a525-4d6c-9be5-eb970bbf517e admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:23.499000 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486121f2-e7a9-40b6-b55a-2ecc53604b9c admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:23.503885 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a2f43d-98f7-48fa-9151-8ff6b73e1979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:23.504489 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2f43d-98f7-48fa-9151-8ff6b73e1979 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:23.504772 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2f43d-98f7-48fa-9151-8ff6b73e1979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:23.505079 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2f43d-98f7-48fa-9151-8ff6b73e1979 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:23.505797 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a2f43d-98f7-48fa-9151-8ff6b73e1979 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:23.507420 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a2f43d-98f7-48fa-9151-8ff6b73e1979 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:23.518739 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 804/1596] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:41:23 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:41:23.519756 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a64373-715b-40ed-a859-8fdc78cb1a3f None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:23.520073 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a64373-715b-40ed-a859-8fdc78cb1a3f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:23.520355 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a64373-715b-40ed-a859-8fdc78cb1a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:23.522036 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 793/1597] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:41:23 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:41:23.522922 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d09209f-a525-4d6c-9be5-eb970bbf517e admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:23.541156 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d6a64373-715b-40ed-a859-8fdc78cb1a3f None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:23.542391 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d6a64373-715b-40ed-a859-8fdc78cb1a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:23.586759 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d09209f-a525-4d6c-9be5-eb970bbf517e admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:23.591447 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e170a94-0f3e-4a58-a801-6f2d07260850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:23.592053 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e170a94-0f3e-4a58-a801-6f2d07260850 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:23.592358 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e170a94-0f3e-4a58-a801-6f2d07260850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:23.592630 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e170a94-0f3e-4a58-a801-6f2d07260850 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:23.593297 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e170a94-0f3e-4a58-a801-6f2d07260850 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:23.594922 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e170a94-0f3e-4a58-a801-6f2d07260850 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:23.603676 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 805/1598] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:23.607898 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 794/1599] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:41:23 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:41:23.607898 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46538cf7-c654-4a2f-b6f7-c92daa6305a6 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:23.607898 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46538cf7-c654-4a2f-b6f7-c92daa6305a6 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:23.607898 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46538cf7-c654-4a2f-b6f7-c92daa6305a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:23.608636 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a64373-715b-40ed-a859-8fdc78cb1a3f None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:23.618372 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a64373-715b-40ed-a859-8fdc78cb1a3f None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:23.623550 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3f683e-f488-481b-aab3-7bb51876e98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:23.624238 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f683e-f488-481b-aab3-7bb51876e98d admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:23.624512 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f683e-f488-481b-aab3-7bb51876e98d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:23.624797 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f683e-f488-481b-aab3-7bb51876e98d admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:23.625425 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3f683e-f488-481b-aab3-7bb51876e98d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:23.627309 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3f683e-f488-481b-aab3-7bb51876e98d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:23.634190 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:41:23.637074 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-46538cf7-c654-4a2f-b6f7-c92daa6305a6 None None] MFA Rules not processed for user `97b222b27b6845e3848db6666db92076`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:23.639152 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-46538cf7-c654-4a2f-b6f7-c92daa6305a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:23.642908 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 806/1600] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:41:23 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:41:23.642908 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3f683e-f488-481b-aab3-7bb51876e98d admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:23.652318 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3f683e-f488-481b-aab3-7bb51876e98d admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:23.657326 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a8f074-2709-4a2a-af7c-2c6f1e281742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:23.657956 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a8f074-2709-4a2a-af7c-2c6f1e281742 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:23.658237 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a8f074-2709-4a2a-af7c-2c6f1e281742 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:23.658507 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a8f074-2709-4a2a-af7c-2c6f1e281742 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:23.659159 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a8f074-2709-4a2a-af7c-2c6f1e281742 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:23.660738 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a8f074-2709-4a2a-af7c-2c6f1e281742 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:23.670586 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 807/1601] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:41:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:41:23.671936 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469b38a-d420-4ecb-a8ea-3fed94666d1f None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:23.672216 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469b38a-d420-4ecb-a8ea-3fed94666d1f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:23.672949 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469b38a-d420-4ecb-a8ea-3fed94666d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:23.691507 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-f469b38a-d420-4ecb-a8ea-3fed94666d1f None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:23.692681 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f469b38a-d420-4ecb-a8ea-3fed94666d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:23.700508 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 808/1602] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:23.701771 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f469b38a-d420-4ecb-a8ea-3fed94666d1f None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:23.713295 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f469b38a-d420-4ecb-a8ea-3fed94666d1f None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:23.715205 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 795/1603] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:23.715205 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46538cf7-c654-4a2f-b6f7-c92daa6305a6 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:23.720135 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b553e747-668c-4230-9c4c-38acb9f2c3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:23.720135 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b553e747-668c-4230-9c4c-38acb9f2c3e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:23.720135 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b553e747-668c-4230-9c4c-38acb9f2c3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:23.720135 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b553e747-668c-4230-9c4c-38acb9f2c3e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:23.720463 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b553e747-668c-4230-9c4c-38acb9f2c3e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:23.721206 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b553e747-668c-4230-9c4c-38acb9f2c3e1 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:23.729559 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46538cf7-c654-4a2f-b6f7-c92daa6305a6 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:23.735565 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 809/1604] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:41:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:41:23.735565 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b553e747-668c-4230-9c4c-38acb9f2c3e1 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:23.737478 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c248952a-9b25-42c0-961d-2bf82ec82826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:23.738201 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248952a-9b25-42c0-961d-2bf82ec82826 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:23.738464 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248952a-9b25-42c0-961d-2bf82ec82826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:23.739607 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248952a-9b25-42c0-961d-2bf82ec82826 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:23.740138 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c248952a-9b25-42c0-961d-2bf82ec82826 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:23.742310 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c248952a-9b25-42c0-961d-2bf82ec82826 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:23.743371 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b553e747-668c-4230-9c4c-38acb9f2c3e1 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:23.749108 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208cbcae-dfa4-4bf7-884c-6c10388e49c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:23.749832 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208cbcae-dfa4-4bf7-884c-6c10388e49c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:23.750339 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208cbcae-dfa4-4bf7-884c-6c10388e49c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:23.750339 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208cbcae-dfa4-4bf7-884c-6c10388e49c3 admin admin] PATH_INFO: `/v3/projects/4eacc288ff1c40c8877635b387ffc3bc/users/f22b26ed7a47480e8448175517170172/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:23.766906 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208cbcae-dfa4-4bf7-884c-6c10388e49c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4eacc288ff1c40c8877635b387ffc3bc, user_id=f22b26ed7a47480e8448175517170172, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:23.769564 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208cbcae-dfa4-4bf7-884c-6c10388e49c3 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:23.774415 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 796/1605] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:41:23 2022] POST /identity/v3/users => generated 297 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:41:23.774415 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c248952a-9b25-42c0-961d-2bf82ec82826 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:23.783932 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 810/1606] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:41:23 2022] PUT /identity/v3/projects/4eacc288ff1c40c8877635b387ffc3bc/users/f22b26ed7a47480e8448175517170172/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:41:23.784814 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208cbcae-dfa4-4bf7-884c-6c10388e49c3 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:23.819399 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:41:23.831064 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c248952a-9b25-42c0-961d-2bf82ec82826 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:23.847481 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208cbcae-dfa4-4bf7-884c-6c10388e49c3 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:23.852084 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c54e23-9f01-400c-ac12-5b8ddcbdf409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:23.852732 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c54e23-9f01-400c-ac12-5b8ddcbdf409 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:23.853011 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c54e23-9f01-400c-ac12-5b8ddcbdf409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:23.853282 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c54e23-9f01-400c-ac12-5b8ddcbdf409 admin admin] PATH_INFO: `/v3/projects/972cdebd72ff41c0aa6a092d6d93693e/users/957373f98691469386b41fe0e17e4bd3/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:23.867325 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c54e23-9f01-400c-ac12-5b8ddcbdf409 admin admin] RBAC: Authorizing `identity:create_grant(project_id=972cdebd72ff41c0aa6a092d6d93693e, user_id=957373f98691469386b41fe0e17e4bd3, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:23.870002 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c54e23-9f01-400c-ac12-5b8ddcbdf409 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:23.881681 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb94011-39ee-44ed-b389-6cd97b8026be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:23.882916 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb94011-39ee-44ed-b389-6cd97b8026be service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:23.883193 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb94011-39ee-44ed-b389-6cd97b8026be service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:23.883460 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb94011-39ee-44ed-b389-6cd97b8026be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:23.884359 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 811/1607] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:41:23 2022] PUT /identity/v3/projects/972cdebd72ff41c0aa6a092d6d93693e/users/957373f98691469386b41fe0e17e4bd3/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:41:23.885144 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c54e23-9f01-400c-ac12-5b8ddcbdf409 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:23.893878 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb94011-39ee-44ed-b389-6cd97b8026be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:23.895727 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb94011-39ee-44ed-b389-6cd97b8026be service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:23.905434 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 797/1608] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:41:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:41:23.906235 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb94011-39ee-44ed-b389-6cd97b8026be service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:23.907625 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7cb94011-39ee-44ed-b389-6cd97b8026be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:23.917953 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb94011-39ee-44ed-b389-6cd97b8026be service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:23.968784 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:41:23.969376 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c54e23-9f01-400c-ac12-5b8ddcbdf409 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:23.971163 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4647b7a-7ce8-4082-b6cc-23ba711b2244 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:23.971792 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4647b7a-7ce8-4082-b6cc-23ba711b2244 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:23.972069 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4647b7a-7ce8-4082-b6cc-23ba711b2244 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:23.972334 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4647b7a-7ce8-4082-b6cc-23ba711b2244 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:23.975159 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdf8baa-58da-4edd-879b-330d6c1bee92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:23.975919 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf8baa-58da-4edd-879b-330d6c1bee92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:23.976205 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf8baa-58da-4edd-879b-330d6c1bee92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:23.976484 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf8baa-58da-4edd-879b-330d6c1bee92 admin admin] PATH_INFO: `/v3/projects/335ecfeb5c6a436e909618c1305593bb/users/298dfca23e004969a0c907e18c7237a4/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:23.991316 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4647b7a-7ce8-4082-b6cc-23ba711b2244 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:23.993649 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4647b7a-7ce8-4082-b6cc-23ba711b2244 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:23.995434 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdf8baa-58da-4edd-879b-330d6c1bee92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=335ecfeb5c6a436e909618c1305593bb, user_id=298dfca23e004969a0c907e18c7237a4, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:23.998955 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdf8baa-58da-4edd-879b-330d6c1bee92 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:24.006876 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 798/1609] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:41:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:41:24.007674 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4647b7a-7ce8-4082-b6cc-23ba711b2244 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:24.018837 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4647b7a-7ce8-4082-b6cc-23ba711b2244 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:24.028149 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 812/1610] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:41:23 2022] PUT /identity/v3/projects/335ecfeb5c6a436e909618c1305593bb/users/298dfca23e004969a0c907e18c7237a4/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:41:24.031383 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1655fc-c8a7-4c9b-9ce8-e86051a6b80d None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:24.032001 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1655fc-c8a7-4c9b-9ce8-e86051a6b80d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:24.033080 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1655fc-c8a7-4c9b-9ce8-e86051a6b80d None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:24.035460 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 813/1611] 10.222.0.52 () {54 vars in 961 bytes} [Wed Sep 28 15:41:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:41:24.036398 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1655fc-c8a7-4c9b-9ce8-e86051a6b80d None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:24.108219 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a407ccd1-eac4-497a-b4dc-99c958c56f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:24.109140 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407ccd1-eac4-497a-b4dc-99c958c56f34 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:24.109417 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407ccd1-eac4-497a-b4dc-99c958c56f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:24.111976 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407ccd1-eac4-497a-b4dc-99c958c56f34 admin admin] PATH_INFO: `/v3/projects/824ddb9f6f344a2684cf31d08427306c/users/36515ae6f91748babdcf6a9a72f59ed1/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:24.130422 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a407ccd1-eac4-497a-b4dc-99c958c56f34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=824ddb9f6f344a2684cf31d08427306c, user_id=36515ae6f91748babdcf6a9a72f59ed1, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:24.133865 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a407ccd1-eac4-497a-b4dc-99c958c56f34 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:24.152810 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 799/1612] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:41:24 2022] PUT /identity/v3/projects/824ddb9f6f344a2684cf31d08427306c/users/36515ae6f91748babdcf6a9a72f59ed1/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:41:24.154341 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c871ddc-d851-4a08-a404-5ca50f549377 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:24.154626 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c871ddc-d851-4a08-a404-5ca50f549377 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:24.154906 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c871ddc-d851-4a08-a404-5ca50f549377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:24.159014 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1655fc-c8a7-4c9b-9ce8-e86051a6b80d None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:24.199876 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-5c871ddc-d851-4a08-a404-5ca50f549377 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:24.201876 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-5c871ddc-d851-4a08-a404-5ca50f549377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:24.221693 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:41:24.250656 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c2806-7728-4624-8ee6-194e6c3c20e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:24.251708 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c2806-7728-4624-8ee6-194e6c3c20e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:24.251992 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c2806-7728-4624-8ee6-194e6c3c20e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:24.252263 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c2806-7728-4624-8ee6-194e6c3c20e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:24.252903 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17c2806-7728-4624-8ee6-194e6c3c20e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:24.254693 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17c2806-7728-4624-8ee6-194e6c3c20e3 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:24.287491 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 814/1613] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:41:24 2022] GET /identity/v3/roles => generated 1420 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:41:24.293764 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e031725-3e56-4792-bd05-bdf354f59991 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:24.293764 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e031725-3e56-4792-bd05-bdf354f59991 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:24.293764 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e031725-3e56-4792-bd05-bdf354f59991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:24.314559 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 800/1614] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:24.316232 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd87368-e627-4c49-92ae-860359730493 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:24.316500 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd87368-e627-4c49-92ae-860359730493 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:24.316793 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd87368-e627-4c49-92ae-860359730493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:24.351248 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-2e031725-3e56-4792-bd05-bdf354f59991 None None] MFA Rules not processed for user `957373f98691469386b41fe0e17e4bd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:24.351745 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-4bd87368-e627-4c49-92ae-860359730493 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:24.353026 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4bd87368-e627-4c49-92ae-860359730493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:24.353648 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2e031725-3e56-4792-bd05-bdf354f59991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:24.367161 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 801/1615] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:24.368976 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e0be17-ed3a-4733-82cb-3f2e0d3a2040 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:24.369655 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e0be17-ed3a-4733-82cb-3f2e0d3a2040 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:24.370192 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e0be17-ed3a-4733-82cb-3f2e0d3a2040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:24.426002 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-a3e0be17-ed3a-4733-82cb-3f2e0d3a2040 None None] MFA Rules not processed for user `8daf14b64bf04b2ba74d2fe1fd41ff71`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:24.426852 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a3e0be17-ed3a-4733-82cb-3f2e0d3a2040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:24.487349 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 815/1616] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:24.488735 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f1e34-940d-4b6c-a00f-1ff5a5feba61 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:24.493358 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f1e34-940d-4b6c-a00f-1ff5a5feba61 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:24.493681 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f1e34-940d-4b6c-a00f-1ff5a5feba61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:24.527622 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:41:24.532184 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-de3f1e34-940d-4b6c-a00f-1ff5a5feba61 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:24.533532 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-de3f1e34-940d-4b6c-a00f-1ff5a5feba61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:24.547988 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 816/1617] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:24.549627 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3f1e34-940d-4b6c-a00f-1ff5a5feba61 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:24.570197 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3f1e34-940d-4b6c-a00f-1ff5a5feba61 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:24.577033 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaab9bfc-3d70-4c48-8be1-35cdf263fc38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:24.577821 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab9bfc-3d70-4c48-8be1-35cdf263fc38 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:24.578223 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab9bfc-3d70-4c48-8be1-35cdf263fc38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:24.578540 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab9bfc-3d70-4c48-8be1-35cdf263fc38 admin admin] PATH_INFO: `/v3/projects/4f0eaf5519ec4454a3e6f6943c0d8486/users/7a31c2863ca54217a709a0227495f7c9/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:24.596655 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 802/1618] 10.222.0.52 () {58 vars in 1058 bytes} [Wed Sep 28 15:41:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:24.597510 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e0be17-ed3a-4733-82cb-3f2e0d3a2040 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:24.606139 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaab9bfc-3d70-4c48-8be1-35cdf263fc38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f0eaf5519ec4454a3e6f6943c0d8486, user_id=7a31c2863ca54217a709a0227495f7c9, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:24.610346 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaab9bfc-3d70-4c48-8be1-35cdf263fc38 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:24.649533 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 817/1619] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:41:24 2022] PUT /identity/v3/projects/4f0eaf5519ec4454a3e6f6943c0d8486/users/7a31c2863ca54217a709a0227495f7c9/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:41:24.650886 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaab9bfc-3d70-4c48-8be1-35cdf263fc38 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:24.773383 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e0be17-ed3a-4733-82cb-3f2e0d3a2040 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:24.796587 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaab9bfc-3d70-4c48-8be1-35cdf263fc38 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:24.803291 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c4b51c-49fd-42b3-9fce-5e1134796dfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:24.804870 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4b51c-49fd-42b3-9fce-5e1134796dfe service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:24.806396 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4b51c-49fd-42b3-9fce-5e1134796dfe service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:24.806901 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4b51c-49fd-42b3-9fce-5e1134796dfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:24.838326 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d305fb8-5208-4bcc-a99b-dfe616b64635 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:24.839989 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d305fb8-5208-4bcc-a99b-dfe616b64635 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:24.840396 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d305fb8-5208-4bcc-a99b-dfe616b64635 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:24.840866 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d305fb8-5208-4bcc-a99b-dfe616b64635 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:24.887018 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:41:24.902817 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c4b51c-49fd-42b3-9fce-5e1134796dfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:24.905485 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c4b51c-49fd-42b3-9fce-5e1134796dfe service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:24.918481 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 818/1620] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:41:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:41:24.919928 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c4b51c-49fd-42b3-9fce-5e1134796dfe service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:24.931558 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c4b51c-49fd-42b3-9fce-5e1134796dfe service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:24.932788 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d305fb8-5208-4bcc-a99b-dfe616b64635 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:24.934533 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d305fb8-5208-4bcc-a99b-dfe616b64635 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:24.938861 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb095a8-93b9-4068-b2cc-2a4cfe83fd2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:24.939963 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb095a8-93b9-4068-b2cc-2a4cfe83fd2e admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:24.940612 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb095a8-93b9-4068-b2cc-2a4cfe83fd2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:24.941110 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb095a8-93b9-4068-b2cc-2a4cfe83fd2e admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:24.942052 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb095a8-93b9-4068-b2cc-2a4cfe83fd2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:24.944624 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb095a8-93b9-4068-b2cc-2a4cfe83fd2e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:24.946742 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 803/1621] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:41:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:41:24.947869 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d305fb8-5208-4bcc-a99b-dfe616b64635 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:24.959649 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d305fb8-5208-4bcc-a99b-dfe616b64635 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:24.965894 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fce9af-d880-452d-a8a7-f553b4554132 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:24.967104 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fce9af-d880-452d-a8a7-f553b4554132 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:24.967379 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fce9af-d880-452d-a8a7-f553b4554132 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:24.967648 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fce9af-d880-452d-a8a7-f553b4554132 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:24.980732 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fce9af-d880-452d-a8a7-f553b4554132 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:24.982635 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fce9af-d880-452d-a8a7-f553b4554132 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:24.988069 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 819/1622] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:41:24 2022] POST /identity/v3/projects => generated 373 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:41:24.989073 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb095a8-93b9-4068-b2cc-2a4cfe83fd2e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:24.996629 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 804/1623] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:41:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:41:24.998152 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fce9af-d880-452d-a8a7-f553b4554132 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:25.023664 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fce9af-d880-452d-a8a7-f553b4554132 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:25.103038 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:41:25.153257 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb095a8-93b9-4068-b2cc-2a4cfe83fd2e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:25.153981 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a28cc2-454a-4435-91e3-df2af9cca49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:25.154607 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a28cc2-454a-4435-91e3-df2af9cca49d admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:25.154880 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a28cc2-454a-4435-91e3-df2af9cca49d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:25.155157 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a28cc2-454a-4435-91e3-df2af9cca49d admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:25.156524 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a28cc2-454a-4435-91e3-df2af9cca49d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:25.160235 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a28cc2-454a-4435-91e3-df2af9cca49d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:25.166149 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e57014b-475f-4908-93ba-ab697b1c707a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf14b64bf04b2ba74d2fe1fd41ff71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:25.166802 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57014b-475f-4908-93ba-ab697b1c707a service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:25.168504 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57014b-475f-4908-93ba-ab697b1c707a service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:25.168789 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57014b-475f-4908-93ba-ab697b1c707a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:25.182295 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 805/1624] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:41:24 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:41:25.182905 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a28cc2-454a-4435-91e3-df2af9cca49d admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:25.204784 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a28cc2-454a-4435-91e3-df2af9cca49d admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:25.218635 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0419e15-bdd5-4a10-9a0a-5a2df20de09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:25.219378 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0419e15-bdd5-4a10-9a0a-5a2df20de09b admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:25.219509 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0419e15-bdd5-4a10-9a0a-5a2df20de09b admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:25.219685 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0419e15-bdd5-4a10-9a0a-5a2df20de09b admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:25.220313 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0419e15-bdd5-4a10-9a0a-5a2df20de09b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:25.222351 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0419e15-bdd5-4a10-9a0a-5a2df20de09b admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:25.265269 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 806/1625] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:41:25 2022] POST /identity/v3/users => generated 293 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:41:25.266339 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0419e15-bdd5-4a10-9a0a-5a2df20de09b admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:25.288665 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0419e15-bdd5-4a10-9a0a-5a2df20de09b admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:25.290200 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e57014b-475f-4908-93ba-ab697b1c707a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:25.293506 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e57014b-475f-4908-93ba-ab697b1c707a service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:25.299996 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2149dbb-59d9-4014-9e42-3aaa8aead29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:25.300729 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2149dbb-59d9-4014-9e42-3aaa8aead29a admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:25.301007 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2149dbb-59d9-4014-9e42-3aaa8aead29a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:25.301286 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2149dbb-59d9-4014-9e42-3aaa8aead29a admin admin] PATH_INFO: `/v3/projects/4f0eaf5519ec4454a3e6f6943c0d8486/users/7a31c2863ca54217a709a0227495f7c9/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:25.306334 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2149dbb-59d9-4014-9e42-3aaa8aead29a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f0eaf5519ec4454a3e6f6943c0d8486, user_id=7a31c2863ca54217a709a0227495f7c9, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:25.311438 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2149dbb-59d9-4014-9e42-3aaa8aead29a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:25.315047 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 820/1626] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:41:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:41:25.316750 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e57014b-475f-4908-93ba-ab697b1c707a service nova] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:25.337306 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 807/1627] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:41:25 2022] PUT /identity/v3/projects/4f0eaf5519ec4454a3e6f6943c0d8486/users/7a31c2863ca54217a709a0227495f7c9/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:41:25.338135 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e57014b-475f-4908-93ba-ab697b1c707a service nova] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:25.349772 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d9250-5950-4782-a628-9f6fea278c2a None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:25.350091 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d9250-5950-4782-a628-9f6fea278c2a None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:25.350391 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d9250-5950-4782-a628-9f6fea278c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:25.409789 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-dd0d9250-5950-4782-a628-9f6fea278c2a None None] MFA Rules not processed for user `7a31c2863ca54217a709a0227495f7c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:25.411812 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-dd0d9250-5950-4782-a628-9f6fea278c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:25.454411 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278e418d-81e5-46b8-b8ce-a78ee4d3c8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:25.455244 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278e418d-81e5-46b8-b8ce-a78ee4d3c8e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:25.455636 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278e418d-81e5-46b8-b8ce-a78ee4d3c8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:25.456112 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278e418d-81e5-46b8-b8ce-a78ee4d3c8e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:25.456937 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278e418d-81e5-46b8-b8ce-a78ee4d3c8e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:25.459199 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278e418d-81e5-46b8-b8ce-a78ee4d3c8e5 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:25.486425 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 821/1628] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:41:25 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:41:25.489525 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278e418d-81e5-46b8-b8ce-a78ee4d3c8e5 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:25.502568 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:41:25.514786 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278e418d-81e5-46b8-b8ce-a78ee4d3c8e5 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:25.528553 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209aeab2-cdd5-48d0-aa90-f1c74fd6a173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:25.531735 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 808/1629] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:25 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:41:25.538402 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209aeab2-cdd5-48d0-aa90-f1c74fd6a173 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:25.538586 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209aeab2-cdd5-48d0-aa90-f1c74fd6a173 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:25.538909 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209aeab2-cdd5-48d0-aa90-f1c74fd6a173 admin admin] PATH_INFO: `/v3/projects/6a0e4b31069447ebb3cf0210cac307d6/users/a7847497073c46afbeaa601617115b06/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:25.540151 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0d9250-5950-4782-a628-9f6fea278c2a None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:25.582728 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209aeab2-cdd5-48d0-aa90-f1c74fd6a173 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a0e4b31069447ebb3cf0210cac307d6, user_id=a7847497073c46afbeaa601617115b06, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:25.588195 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209aeab2-cdd5-48d0-aa90-f1c74fd6a173 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:25.619233 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 822/1630] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:41:25 2022] PUT /identity/v3/projects/6a0e4b31069447ebb3cf0210cac307d6/users/a7847497073c46afbeaa601617115b06/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:41:25.623794 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209aeab2-cdd5-48d0-aa90-f1c74fd6a173 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:25.666887 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0d9250-5950-4782-a628-9f6fea278c2a None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:25.721091 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:41:25.733719 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d48071-d99c-466e-8fcf-eefaeb1aff7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:25.734394 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d48071-d99c-466e-8fcf-eefaeb1aff7c service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:25.734669 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d48071-d99c-466e-8fcf-eefaeb1aff7c service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:25.734939 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d48071-d99c-466e-8fcf-eefaeb1aff7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:25.789241 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209aeab2-cdd5-48d0-aa90-f1c74fd6a173 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:25.801601 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb6f9b1-eb1c-474b-aac2-e662015dcdc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:25.802286 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6f9b1-eb1c-474b-aac2-e662015dcdc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:25.802523 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6f9b1-eb1c-474b-aac2-e662015dcdc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:25.802799 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6f9b1-eb1c-474b-aac2-e662015dcdc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:25.806000 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb6f9b1-eb1c-474b-aac2-e662015dcdc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:25.808351 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb6f9b1-eb1c-474b-aac2-e662015dcdc1 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:25.819191 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d48071-d99c-466e-8fcf-eefaeb1aff7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:25.823012 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d48071-d99c-466e-8fcf-eefaeb1aff7c service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:25.836238 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 809/1631] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:41:25 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:41:25.837888 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d48071-d99c-466e-8fcf-eefaeb1aff7c service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:25.847639 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 823/1632] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:41:25 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:41:25.864365 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d48071-d99c-466e-8fcf-eefaeb1aff7c service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:25.877352 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44359878-870a-4f8f-9334-c47d156ca2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:25.879284 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44359878-870a-4f8f-9334-c47d156ca2df admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:25.881155 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44359878-870a-4f8f-9334-c47d156ca2df admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:25.881512 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44359878-870a-4f8f-9334-c47d156ca2df admin admin] PATH_INFO: `/v3/projects/6a0e4b31069447ebb3cf0210cac307d6/users/a7847497073c46afbeaa601617115b06/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:25.895368 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44359878-870a-4f8f-9334-c47d156ca2df admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a0e4b31069447ebb3cf0210cac307d6, user_id=a7847497073c46afbeaa601617115b06, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:25.903137 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44359878-870a-4f8f-9334-c47d156ca2df admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:25.930564 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 810/1633] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:41:25 2022] PUT /identity/v3/projects/6a0e4b31069447ebb3cf0210cac307d6/users/a7847497073c46afbeaa601617115b06/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:41:25.947050 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37262c57-1b97-49c4-8e99-045862b92462 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:25.947350 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37262c57-1b97-49c4-8e99-045862b92462 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:25.947633 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37262c57-1b97-49c4-8e99-045862b92462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:26.046312 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-37262c57-1b97-49c4-8e99-045862b92462 None None] MFA Rules not processed for user `a7847497073c46afbeaa601617115b06`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:26.058993 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-37262c57-1b97-49c4-8e99-045862b92462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:26.082465 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44359878-870a-4f8f-9334-c47d156ca2df admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:26.210689 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44359878-870a-4f8f-9334-c47d156ca2df admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:26.219601 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec24e77-a46d-49ff-8ec6-aa295eecd811 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:26.220241 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec24e77-a46d-49ff-8ec6-aa295eecd811 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:26.220519 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec24e77-a46d-49ff-8ec6-aa295eecd811 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:26.220786 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec24e77-a46d-49ff-8ec6-aa295eecd811 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:26.249200 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 824/1634] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:25 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:26.253359 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37262c57-1b97-49c4-8e99-045862b92462 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:26.265209 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37262c57-1b97-49c4-8e99-045862b92462 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:26.269467 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e471ffa-dfdc-4c25-a821-dea8f469e469 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:26.274633 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e471ffa-dfdc-4c25-a821-dea8f469e469 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:26.275236 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e471ffa-dfdc-4c25-a821-dea8f469e469 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:26.275800 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e471ffa-dfdc-4c25-a821-dea8f469e469 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:26.318691 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e471ffa-dfdc-4c25-a821-dea8f469e469 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:26.323757 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e471ffa-dfdc-4c25-a821-dea8f469e469 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:26.369316 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 825/1635] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:41:26 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:41:26.406481 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec24e77-a46d-49ff-8ec6-aa295eecd811 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:26.407968 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec24e77-a46d-49ff-8ec6-aa295eecd811 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:26.419569 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 811/1636] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:41:26 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:41:26.473029 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfd9120-a2cd-4cad-adf4-829c5eca05f0 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:26.473328 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfd9120-a2cd-4cad-adf4-829c5eca05f0 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:26.473792 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfd9120-a2cd-4cad-adf4-829c5eca05f0 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:26.475548 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 826/1637] 10.222.0.52 () {54 vars in 961 bytes} [Wed Sep 28 15:41:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:41:26.500701 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de5523-b7b1-4b53-81b8-5f8b661feda4 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:26.501638 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de5523-b7b1-4b53-81b8-5f8b661feda4 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:26.502109 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de5523-b7b1-4b53-81b8-5f8b661feda4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:26.550577 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-34de5523-b7b1-4b53-81b8-5f8b661feda4 None None] MFA Rules not processed for user `8daf14b64bf04b2ba74d2fe1fd41ff71`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:26.551842 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfd9120-a2cd-4cad-adf4-829c5eca05f0 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:26.564975 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-34de5523-b7b1-4b53-81b8-5f8b661feda4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:26.571096 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfd9120-a2cd-4cad-adf4-829c5eca05f0 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:26.576189 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb061a1-dca2-4dd7-a83d-b4321aa5e636 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:26.577744 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb061a1-dca2-4dd7-a83d-b4321aa5e636 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:26.579338 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb061a1-dca2-4dd7-a83d-b4321aa5e636 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:26.579876 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb061a1-dca2-4dd7-a83d-b4321aa5e636 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:26.657874 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:41:26.662970 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:41:26.695366 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb061a1-dca2-4dd7-a83d-b4321aa5e636 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:26.697702 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb061a1-dca2-4dd7-a83d-b4321aa5e636 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:26.725371 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 827/1638] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:41:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:41:26.780137 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 812/1639] 10.222.0.52 () {58 vars in 1058 bytes} [Wed Sep 28 15:41:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:26.780824 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d12e3-14ac-42d2-b366-5f22458ff570 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:26.781115 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d12e3-14ac-42d2-b366-5f22458ff570 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:26.781388 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d12e3-14ac-42d2-b366-5f22458ff570 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:26.782939 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 828/1640] 10.222.0.52 () {54 vars in 912 bytes} [Wed Sep 28 15:41:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:41:26.790732 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df7951-93f7-403e-8ffd-cb66eb7f0918 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:26.792742 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df7951-93f7-403e-8ffd-cb66eb7f0918 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:26.793705 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df7951-93f7-403e-8ffd-cb66eb7f0918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:26.842597 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-89df7951-93f7-403e-8ffd-cb66eb7f0918 None None] MFA Rules not processed for user `8daf14b64bf04b2ba74d2fe1fd41ff71`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:26.844260 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-89df7951-93f7-403e-8ffd-cb66eb7f0918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:26.869551 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 813/1641] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Sep 28 15:41:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:27.007610 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5d12e3-14ac-42d2-b366-5f22458ff570 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:27.026958 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5d12e3-14ac-42d2-b366-5f22458ff570 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:27.036740 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eab1fd-1f46-44ac-8f85-bff1076a9695 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:27.040088 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eab1fd-1f46-44ac-8f85-bff1076a9695 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:27.042114 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eab1fd-1f46-44ac-8f85-bff1076a9695 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:27.043785 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eab1fd-1f46-44ac-8f85-bff1076a9695 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:27.100525 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eab1fd-1f46-44ac-8f85-bff1076a9695 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:27.102048 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eab1fd-1f46-44ac-8f85-bff1076a9695 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:27.158890 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 829/1642] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:41:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:41:27.471352 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7838c0-8ae1-48b9-b0fd-6318a4cdef68 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:27.473034 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7838c0-8ae1-48b9-b0fd-6318a4cdef68 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:27.473420 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7838c0-8ae1-48b9-b0fd-6318a4cdef68 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:27.475234 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 814/1643] 10.222.0.52 () {54 vars in 912 bytes} [Wed Sep 28 15:41:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:41:27.485332 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb4b0b-9fc1-4049-a7ef-f7b10ef1165f None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:27.485614 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb4b0b-9fc1-4049-a7ef-f7b10ef1165f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:27.485944 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb4b0b-9fc1-4049-a7ef-f7b10ef1165f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:27.523869 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-c2cb4b0b-9fc1-4049-a7ef-f7b10ef1165f None None] MFA Rules not processed for user `8daf14b64bf04b2ba74d2fe1fd41ff71`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:27.527402 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c2cb4b0b-9fc1-4049-a7ef-f7b10ef1165f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:27.546178 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 830/1644] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Sep 28 15:41:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:27.561384 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7838c0-8ae1-48b9-b0fd-6318a4cdef68 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:27.579908 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7838c0-8ae1-48b9-b0fd-6318a4cdef68 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:27.587662 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffbb453-7fe1-41ce-b299-b120ecaa0d9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:27.588920 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffbb453-7fe1-41ce-b299-b120ecaa0d9f service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:27.590147 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffbb453-7fe1-41ce-b299-b120ecaa0d9f service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:27.590431 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffbb453-7fe1-41ce-b299-b120ecaa0d9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:27.618967 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffbb453-7fe1-41ce-b299-b120ecaa0d9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:27.620424 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffbb453-7fe1-41ce-b299-b120ecaa0d9f service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:27.659318 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 815/1645] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:41:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:41:27.858028 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb4b0b-9fc1-4049-a7ef-f7b10ef1165f None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:27.999349 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb4b0b-9fc1-4049-a7ef-f7b10ef1165f None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:28.005644 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd2fe3-cd3c-4f61-88a0-c40bca1dc79c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:28.006887 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd2fe3-cd3c-4f61-88a0-c40bca1dc79c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:28.008159 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd2fe3-cd3c-4f61-88a0-c40bca1dc79c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:28.009149 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd2fe3-cd3c-4f61-88a0-c40bca1dc79c admin admin] PATH_INFO: `/v3/users/97b222b27b6845e3848db6666db92076` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:28.012079 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadd2fe3-cd3c-4f61-88a0-c40bca1dc79c admin admin] RBAC: Authorizing `identity:delete_user(user_id=97b222b27b6845e3848db6666db92076)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:28.015176 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadd2fe3-cd3c-4f61-88a0-c40bca1dc79c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:28.135164 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-aadd2fe3-cd3c-4f61-88a0-c40bca1dc79c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97b222b27b6845e3848db6666db92076'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:28.139347 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-aadd2fe3-cd3c-4f61-88a0-c40bca1dc79c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97b222b27b6845e3848db6666db92076'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:28.190035 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-aadd2fe3-cd3c-4f61-88a0-c40bca1dc79c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97b222b27b6845e3848db6666db92076'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:28.214892 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-aadd2fe3-cd3c-4f61-88a0-c40bca1dc79c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97b222b27b6845e3848db6666db92076'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:28.240529 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-aadd2fe3-cd3c-4f61-88a0-c40bca1dc79c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97b222b27b6845e3848db6666db92076'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:28.255901 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 831/1646] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:41:27 2022] DELETE /identity/v3/users/97b222b27b6845e3848db6666db92076 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:41:28.260299 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b57bb72-7e08-463d-984f-9f43ce1dbcda None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:28.260860 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b57bb72-7e08-463d-984f-9f43ce1dbcda None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:28.261129 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b57bb72-7e08-463d-984f-9f43ce1dbcda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:28.318178 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-6b57bb72-7e08-463d-984f-9f43ce1dbcda None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:28.320866 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-6b57bb72-7e08-463d-984f-9f43ce1dbcda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:28.521349 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 816/1647] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:28.581090 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd2fe3-cd3c-4f61-88a0-c40bca1dc79c admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:28.584190 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-aadd2fe3-cd3c-4f61-88a0-c40bca1dc79c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:28.760585 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:41:28.798154 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd2fe3-cd3c-4f61-88a0-c40bca1dc79c admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:28.805128 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3aeab3-a12d-4a1b-ad30-863359f4f4ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:28.805923 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3aeab3-a12d-4a1b-ad30-863359f4f4ee service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:28.806211 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3aeab3-a12d-4a1b-ad30-863359f4f4ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:28.806485 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3aeab3-a12d-4a1b-ad30-863359f4f4ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:28.850366 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3aeab3-a12d-4a1b-ad30-863359f4f4ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:28.851961 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3aeab3-a12d-4a1b-ad30-863359f4f4ee service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:28.881509 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 832/1648] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:41:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:41:30.502203 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b57bb72-7e08-463d-984f-9f43ce1dbcda None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:30.504270 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-6b57bb72-7e08-463d-984f-9f43ce1dbcda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:30.523160 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b57bb72-7e08-463d-984f-9f43ce1dbcda None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:30.539891 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a386cba5-da2d-4fa6-9c1e-f61c0d2a70d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:30.540545 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a386cba5-da2d-4fa6-9c1e-f61c0d2a70d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:30.540823 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a386cba5-da2d-4fa6-9c1e-f61c0d2a70d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:30.541093 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a386cba5-da2d-4fa6-9c1e-f61c0d2a70d1 admin admin] PATH_INFO: `/v3/projects/2af27dac96724c4592f66af8801ff80b` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:30.542388 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a386cba5-da2d-4fa6-9c1e-f61c0d2a70d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2af27dac96724c4592f66af8801ff80b)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:30.544992 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a386cba5-da2d-4fa6-9c1e-f61c0d2a70d1 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:30.653956 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a386cba5-da2d-4fa6-9c1e-f61c0d2a70d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2af27dac96724c4592f66af8801ff80b', 'request_id': 'req-a386cba5-da2d-4fa6-9c1e-f61c0d2a70d1'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:30.682896 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a386cba5-da2d-4fa6-9c1e-f61c0d2a70d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2af27dac96724c4592f66af8801ff80b', 'request_id': 'req-a386cba5-da2d-4fa6-9c1e-f61c0d2a70d1'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:30.689892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a386cba5-da2d-4fa6-9c1e-f61c0d2a70d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2af27dac96724c4592f66af8801ff80b', 'request_id': 'req-a386cba5-da2d-4fa6-9c1e-f61c0d2a70d1'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:30.716745 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a386cba5-da2d-4fa6-9c1e-f61c0d2a70d1 admin admin] The token cache is being invalidate because project 2af27dac96724c4592f66af8801ff80b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:41:30.717052 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a386cba5-da2d-4fa6-9c1e-f61c0d2a70d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:30.717726 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a386cba5-da2d-4fa6-9c1e-f61c0d2a70d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:30.726475 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 817/1649] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:41:30 2022] DELETE /identity/v3/projects/2af27dac96724c4592f66af8801ff80b => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:41:30.728063 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bd00d2-6fb5-4bd1-9a81-d7f2651ff3c5 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:30.728543 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bd00d2-6fb5-4bd1-9a81-d7f2651ff3c5 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:30.728617 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bd00d2-6fb5-4bd1-9a81-d7f2651ff3c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:30.787238 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-90bd00d2-6fb5-4bd1-9a81-d7f2651ff3c5 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:30.788390 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-90bd00d2-6fb5-4bd1-9a81-d7f2651ff3c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:31.003951 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 833/1650] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:31.005662 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d5b962-cd7e-4cbc-87f7-0959da08c2d8 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:31.005976 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d5b962-cd7e-4cbc-87f7-0959da08c2d8 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:31.006255 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d5b962-cd7e-4cbc-87f7-0959da08c2d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:31.069036 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-41d5b962-cd7e-4cbc-87f7-0959da08c2d8 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:31.076268 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-41d5b962-cd7e-4cbc-87f7-0959da08c2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:31.104839 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 818/1651] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:31.109296 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bd00d2-6fb5-4bd1-9a81-d7f2651ff3c5 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:31.120198 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bd00d2-6fb5-4bd1-9a81-d7f2651ff3c5 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:31.139241 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e27c15a-6528-45e6-858b-2991d678c647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:31.139915 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27c15a-6528-45e6-858b-2991d678c647 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:31.140167 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27c15a-6528-45e6-858b-2991d678c647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:31.140931 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27c15a-6528-45e6-858b-2991d678c647 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:31.141839 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e27c15a-6528-45e6-858b-2991d678c647 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:41:31.143087 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e27c15a-6528-45e6-858b-2991d678c647 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:31.145374 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e27c15a-6528-45e6-858b-2991d678c647 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:31.160276 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 834/1652] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:41:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:41:31.165080 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d5b962-cd7e-4cbc-87f7-0959da08c2d8 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:31.184214 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d5b962-cd7e-4cbc-87f7-0959da08c2d8 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:31.201365 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafe449e-cbde-4cca-abdc-b70c3e4baec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:31.201595 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe449e-cbde-4cca-abdc-b70c3e4baec8 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:31.201821 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe449e-cbde-4cca-abdc-b70c3e4baec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:31.202115 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe449e-cbde-4cca-abdc-b70c3e4baec8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:31.202933 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafe449e-cbde-4cca-abdc-b70c3e4baec8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:31.205057 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafe449e-cbde-4cca-abdc-b70c3e4baec8 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:31.261966 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:41:31.285642 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e27c15a-6528-45e6-858b-2991d678c647 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:31.287796 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 819/1653] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:41:31 2022] POST /identity/v3/projects => generated 405 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:41:31.300645 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e27c15a-6528-45e6-858b-2991d678c647 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:31.513536 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3546cd6d-e05c-4446-9f09-5325f3786e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:31.514211 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3546cd6d-e05c-4446-9f09-5325f3786e65 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:31.514492 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3546cd6d-e05c-4446-9f09-5325f3786e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:31.514763 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3546cd6d-e05c-4446-9f09-5325f3786e65 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:31.515401 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3546cd6d-e05c-4446-9f09-5325f3786e65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:31.517338 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3546cd6d-e05c-4446-9f09-5325f3786e65 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:31.558199 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafe449e-cbde-4cca-abdc-b70c3e4baec8 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:31.572050 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 835/1654] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:41:31 2022] POST /identity/v3/users => generated 303 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:41:31.576609 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafe449e-cbde-4cca-abdc-b70c3e4baec8 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:31.589449 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02723540-445b-4bd3-ac46-419d999552e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:31.591133 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02723540-445b-4bd3-ac46-419d999552e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:31.591647 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02723540-445b-4bd3-ac46-419d999552e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:31.591969 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02723540-445b-4bd3-ac46-419d999552e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:31.592625 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02723540-445b-4bd3-ac46-419d999552e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:31.606763 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02723540-445b-4bd3-ac46-419d999552e3 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:31.635677 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3546cd6d-e05c-4446-9f09-5325f3786e65 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:31.664445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 820/1655] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:41:31.669523 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3546cd6d-e05c-4446-9f09-5325f3786e65 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:31.689308 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dddbdc4-0995-4846-bb0b-dfee10541660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:31.690105 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dddbdc4-0995-4846-bb0b-dfee10541660 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:31.691975 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dddbdc4-0995-4846-bb0b-dfee10541660 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:31.692363 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dddbdc4-0995-4846-bb0b-dfee10541660 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:31.693257 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dddbdc4-0995-4846-bb0b-dfee10541660 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:31.697325 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dddbdc4-0995-4846-bb0b-dfee10541660 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:31.728638 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 836/1656] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:41:31.753026 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02723540-445b-4bd3-ac46-419d999552e3 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:31.767815 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02723540-445b-4bd3-ac46-419d999552e3 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:31.779799 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f2d41-46c9-4a3b-8e92-dfc9cfee6868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:31.780309 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f2d41-46c9-4a3b-8e92-dfc9cfee6868 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:31.781306 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f2d41-46c9-4a3b-8e92-dfc9cfee6868 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:31.782889 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f2d41-46c9-4a3b-8e92-dfc9cfee6868 admin admin] PATH_INFO: `/v3/projects/d7287e9f86c64792bcdf362bba013439/users/5108b5cf3d3948379ccefa22fe573784/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:31.852682 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08f2d41-46c9-4a3b-8e92-dfc9cfee6868 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7287e9f86c64792bcdf362bba013439, user_id=5108b5cf3d3948379ccefa22fe573784, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:31.860124 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08f2d41-46c9-4a3b-8e92-dfc9cfee6868 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:31.892190 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 821/1657] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:41:31 2022] PUT /identity/v3/projects/d7287e9f86c64792bcdf362bba013439/users/5108b5cf3d3948379ccefa22fe573784/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:41:31.894097 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a0ade7-d354-47e7-b99b-8bef118ea825 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:31.894450 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a0ade7-d354-47e7-b99b-8bef118ea825 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:31.894727 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a0ade7-d354-47e7-b99b-8bef118ea825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:31.936141 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d7a0ade7-d354-47e7-b99b-8bef118ea825 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:31.937474 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d7a0ade7-d354-47e7-b99b-8bef118ea825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:32.052191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:41:32.099182 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 837/1658] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:32.129439 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f2d41-46c9-4a3b-8e92-dfc9cfee6868 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:32.132262 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b08f2d41-46c9-4a3b-8e92-dfc9cfee6868 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:32.259194 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f2d41-46c9-4a3b-8e92-dfc9cfee6868 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:32.265327 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4310195-797e-4f32-a4d3-fdd2608daa80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:32.266433 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4310195-797e-4f32-a4d3-fdd2608daa80 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:32.266846 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4310195-797e-4f32-a4d3-fdd2608daa80 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:32.267358 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4310195-797e-4f32-a4d3-fdd2608daa80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:32.306491 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4310195-797e-4f32-a4d3-fdd2608daa80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:32.310437 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4310195-797e-4f32-a4d3-fdd2608daa80 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:32.327302 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 822/1659] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:41:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:41:34.467870 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa72a124-871e-47cd-8661-aba94eb08370 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:34.468389 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa72a124-871e-47cd-8661-aba94eb08370 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:34.469295 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa72a124-871e-47cd-8661-aba94eb08370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:34.603366 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-fa72a124-871e-47cd-8661-aba94eb08370 None None] MFA Rules not processed for user `5108b5cf3d3948379ccefa22fe573784`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:34.608985 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-fa72a124-871e-47cd-8661-aba94eb08370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:34.782322 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 838/1660] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:34 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:34.782871 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4310195-797e-4f32-a4d3-fdd2608daa80 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:34.793099 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4310195-797e-4f32-a4d3-fdd2608daa80 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:34.799803 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cca409-14c6-49bc-a7e4-fbdeb5c17c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:34.801086 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cca409-14c6-49bc-a7e4-fbdeb5c17c30 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:34.801515 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cca409-14c6-49bc-a7e4-fbdeb5c17c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:34.801995 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cca409-14c6-49bc-a7e4-fbdeb5c17c30 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:34.802812 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cca409-14c6-49bc-a7e4-fbdeb5c17c30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:34.805301 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cca409-14c6-49bc-a7e4-fbdeb5c17c30 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:34.858925 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 823/1661] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:41:34 2022] POST /identity/v3/projects => generated 405 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:41:34.861881 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa72a124-871e-47cd-8661-aba94eb08370 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:34.887142 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa72a124-871e-47cd-8661-aba94eb08370 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:35.076294 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0fb022-00a8-468e-ac31-a554fc4f0987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:35.081893 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0fb022-00a8-468e-ac31-a554fc4f0987 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:35.082890 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0fb022-00a8-468e-ac31-a554fc4f0987 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:35.083385 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0fb022-00a8-468e-ac31-a554fc4f0987 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:35.095132 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0fb022-00a8-468e-ac31-a554fc4f0987 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:35.099804 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0fb022-00a8-468e-ac31-a554fc4f0987 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:35.150443 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 839/1662] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:41:34 2022] POST /identity/v3/users => generated 309 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:41:35.152281 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cca409-14c6-49bc-a7e4-fbdeb5c17c30 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:35.178022 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cca409-14c6-49bc-a7e4-fbdeb5c17c30 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:35.186223 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b9b15-e257-4255-9de8-6c09b954a930 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:35.186513 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b9b15-e257-4255-9de8-6c09b954a930 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:35.186793 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b9b15-e257-4255-9de8-6c09b954a930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:35.187586 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f080adb9-ad58-4c62-8e89-3df500546a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:35.190435 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080adb9-ad58-4c62-8e89-3df500546a18 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:35.190917 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080adb9-ad58-4c62-8e89-3df500546a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:35.191418 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080adb9-ad58-4c62-8e89-3df500546a18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:35.192235 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f080adb9-ad58-4c62-8e89-3df500546a18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:35.196153 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f080adb9-ad58-4c62-8e89-3df500546a18 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:35.210591 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 824/1663] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:41:35 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:41:35.214487 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f080adb9-ad58-4c62-8e89-3df500546a18 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:35.228220 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f080adb9-ad58-4c62-8e89-3df500546a18 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:35.234847 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7844f8e-1ec9-4709-92ba-cd43b9a8bd47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:35.235755 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7844f8e-1ec9-4709-92ba-cd43b9a8bd47 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:35.236938 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7844f8e-1ec9-4709-92ba-cd43b9a8bd47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:35.237425 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7844f8e-1ec9-4709-92ba-cd43b9a8bd47 admin admin] PATH_INFO: `/v3/projects/fd72bda4e1944528a2f342e3a25173d0/users/715c52a2ce0742d19a9c5b04cdd536e1/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:35.252086 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-984b9b15-e257-4255-9de8-6c09b954a930 None None] MFA Rules not processed for user `f22b26ed7a47480e8448175517170172`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:35.255391 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-984b9b15-e257-4255-9de8-6c09b954a930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:35.266185 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7844f8e-1ec9-4709-92ba-cd43b9a8bd47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd72bda4e1944528a2f342e3a25173d0, user_id=715c52a2ce0742d19a9c5b04cdd536e1, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:35.280667 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7844f8e-1ec9-4709-92ba-cd43b9a8bd47 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:35.323865 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 825/1664] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:41:35 2022] PUT /identity/v3/projects/fd72bda4e1944528a2f342e3a25173d0/users/715c52a2ce0742d19a9c5b04cdd536e1/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:41:35.329038 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7844f8e-1ec9-4709-92ba-cd43b9a8bd47 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:35.444379 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 840/1665] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:41:35.455820 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b9b15-e257-4255-9de8-6c09b954a930 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:35.457324 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-984b9b15-e257-4255-9de8-6c09b954a930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:35.465428 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:41:35.489242 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7844f8e-1ec9-4709-92ba-cd43b9a8bd47 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:35.505335 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d147865-ae87-483a-804d-504d8febda25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:35.506966 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d147865-ae87-483a-804d-504d8febda25 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:35.527765 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d147865-ae87-483a-804d-504d8febda25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:35.534886 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d147865-ae87-483a-804d-504d8febda25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:35.535714 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d147865-ae87-483a-804d-504d8febda25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:35.538636 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d147865-ae87-483a-804d-504d8febda25 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:35.565259 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 826/1666] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:41:35 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:41:35.568648 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d147865-ae87-483a-804d-504d8febda25 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:35.599205 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d147865-ae87-483a-804d-504d8febda25 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:35.617786 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52cd043-b3f2-42c9-8350-69a7324dc5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:35.619961 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52cd043-b3f2-42c9-8350-69a7324dc5a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:35.622821 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52cd043-b3f2-42c9-8350-69a7324dc5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:35.624915 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52cd043-b3f2-42c9-8350-69a7324dc5a9 admin admin] PATH_INFO: `/v3/projects/fd72bda4e1944528a2f342e3a25173d0/users/715c52a2ce0742d19a9c5b04cdd536e1/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:35.632119 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52cd043-b3f2-42c9-8350-69a7324dc5a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd72bda4e1944528a2f342e3a25173d0, user_id=715c52a2ce0742d19a9c5b04cdd536e1, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:35.645836 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52cd043-b3f2-42c9-8350-69a7324dc5a9 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:35.655272 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:41:35.689338 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b9b15-e257-4255-9de8-6c09b954a930 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:35.695759 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 827/1667] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:41:35 2022] PUT /identity/v3/projects/fd72bda4e1944528a2f342e3a25173d0/users/715c52a2ce0742d19a9c5b04cdd536e1/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:41:35.873624 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b345b972-a39e-409a-91cf-a261882f4842 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf14b64bf04b2ba74d2fe1fd41ff71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:35.874679 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345b972-a39e-409a-91cf-a261882f4842 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:35.875111 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345b972-a39e-409a-91cf-a261882f4842 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:35.875522 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345b972-a39e-409a-91cf-a261882f4842 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:36.047034 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b345b972-a39e-409a-91cf-a261882f4842 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:36.048891 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b345b972-a39e-409a-91cf-a261882f4842 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:36.082980 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 841/1668] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:41:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 628 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:41:39.614923 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802c189-6053-4506-8a0f-c816427a0d6b None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:39.615911 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802c189-6053-4506-8a0f-c816427a0d6b None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:39.616386 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802c189-6053-4506-8a0f-c816427a0d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:39.689096 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-c802c189-6053-4506-8a0f-c816427a0d6b None None] MFA Rules not processed for user `715c52a2ce0742d19a9c5b04cdd536e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:39.695467 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c802c189-6053-4506-8a0f-c816427a0d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:39.958446 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 828/1669] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:39 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:41.072103 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f8f52-8d55-468f-a56c-858a9e1bd667 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:41.072604 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f8f52-8d55-468f-a56c-858a9e1bd667 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:41.072604 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f8f52-8d55-468f-a56c-858a9e1bd667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:41.157999 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-836f8f52-8d55-468f-a56c-858a9e1bd667 None None] MFA Rules not processed for user `634c0af05b334b488f2fa358e574172b`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:41.160334 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-836f8f52-8d55-468f-a56c-858a9e1bd667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:41.191947 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c802c189-6053-4506-8a0f-c816427a0d6b None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:41.194489 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c802c189-6053-4506-8a0f-c816427a0d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:41.256672 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:41:41.370148 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c802c189-6053-4506-8a0f-c816427a0d6b None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:41.377193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 842/1670] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:41:41.383836 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a673d77-17f9-4f55-bf92-a8274d35d4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:41.385132 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a673d77-17f9-4f55-bf92-a8274d35d4e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:41.385643 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a673d77-17f9-4f55-bf92-a8274d35d4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:41.387124 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a673d77-17f9-4f55-bf92-a8274d35d4e8 admin admin] PATH_INFO: `/v3/users/50450686eb4d45e5828d186c6ea8a515` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:41.391870 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836f8f52-8d55-468f-a56c-858a9e1bd667 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:41.394004 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-836f8f52-8d55-468f-a56c-858a9e1bd667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:41.400942 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a673d77-17f9-4f55-bf92-a8274d35d4e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50450686eb4d45e5828d186c6ea8a515)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:41.404735 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: WARNING py.warnings [None req-2a673d77-17f9-4f55-bf92-a8274d35d4e8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 15:41:41.404735 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: warnings.warn(msg) Sep 28 15:41:41.404735 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: Sep 28 15:41:41.406041 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a673d77-17f9-4f55-bf92-a8274d35d4e8 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:41.445575 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836f8f52-8d55-468f-a56c-858a9e1bd667 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:41.469276 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce3b32-b354-4f6b-9996-55646b3be61e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf14b64bf04b2ba74d2fe1fd41ff71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:41.472279 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce3b32-b354-4f6b-9996-55646b3be61e service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:41.472614 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce3b32-b354-4f6b-9996-55646b3be61e service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:41.474655 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce3b32-b354-4f6b-9996-55646b3be61e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:41.501626 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ce3b32-b354-4f6b-9996-55646b3be61e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:41.521800 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ce3b32-b354-4f6b-9996-55646b3be61e service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:41.548397 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 843/1671] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:41:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:41:41.612134 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-2a673d77-17f9-4f55-bf92-a8274d35d4e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50450686eb4d45e5828d186c6ea8a515'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:41.639569 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-2a673d77-17f9-4f55-bf92-a8274d35d4e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50450686eb4d45e5828d186c6ea8a515'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:41.640343 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-2a673d77-17f9-4f55-bf92-a8274d35d4e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50450686eb4d45e5828d186c6ea8a515'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:41.659922 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-2a673d77-17f9-4f55-bf92-a8274d35d4e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50450686eb4d45e5828d186c6ea8a515'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:41.661958 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-2a673d77-17f9-4f55-bf92-a8274d35d4e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50450686eb4d45e5828d186c6ea8a515'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:41.724875 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:41:41.738611 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 829/1672] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:41:41 2022] DELETE /identity/v3/users/50450686eb4d45e5828d186c6ea8a515 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:41:41.743876 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1fa13-b4a7-4c45-960d-cbdd8554a323 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:41.744241 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1fa13-b4a7-4c45-960d-cbdd8554a323 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:41.744589 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1fa13-b4a7-4c45-960d-cbdd8554a323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:41.767498 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-edf1fa13-b4a7-4c45-960d-cbdd8554a323 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:41.771754 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-edf1fa13-b4a7-4c45-960d-cbdd8554a323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:41.916724 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 844/1673] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:41.951382 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a673d77-17f9-4f55-bf92-a8274d35d4e8 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:41.953052 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-2a673d77-17f9-4f55-bf92-a8274d35d4e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:42.071954 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a673d77-17f9-4f55-bf92-a8274d35d4e8 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:42.079970 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc985b79-df2f-4a8e-a4ac-918b7b0fe744 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:42.082258 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc985b79-df2f-4a8e-a4ac-918b7b0fe744 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:42.082677 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc985b79-df2f-4a8e-a4ac-918b7b0fe744 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:42.083110 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc985b79-df2f-4a8e-a4ac-918b7b0fe744 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:42.125665 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc985b79-df2f-4a8e-a4ac-918b7b0fe744 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:42.128025 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc985b79-df2f-4a8e-a4ac-918b7b0fe744 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:42.160409 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 830/1674] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:41:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:41:42.982562 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41d869-54b7-4856-8737-442e48f656f0 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:42.983098 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41d869-54b7-4856-8737-442e48f656f0 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:42.983575 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41d869-54b7-4856-8737-442e48f656f0 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:42.985386 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 845/1675] 10.222.0.52 () {54 vars in 921 bytes} [Wed Sep 28 15:41:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:41:43.028451 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a346992-a9ba-4d8d-9995-0846a96cd0ba None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:43.029501 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a346992-a9ba-4d8d-9995-0846a96cd0ba None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:43.029956 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a346992-a9ba-4d8d-9995-0846a96cd0ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:43.082604 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-3a346992-a9ba-4d8d-9995-0846a96cd0ba None None] MFA Rules not processed for user `de934d0105bd4d01ba6c06a4a0d36704`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:43.108586 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3a346992-a9ba-4d8d-9995-0846a96cd0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:43.269003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 831/1676] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Sep 28 15:41:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:41:43.453874 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c41d869-54b7-4856-8737-442e48f656f0 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:43.456765 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1c41d869-54b7-4856-8737-442e48f656f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:43.507003 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c41d869-54b7-4856-8737-442e48f656f0 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:43.511045 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c7ea2-eff0-42c4-a0fb-80abd35fec77 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:43.511329 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c7ea2-eff0-42c4-a0fb-80abd35fec77 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:43.511598 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c7ea2-eff0-42c4-a0fb-80abd35fec77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:43.522267 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8652b1-ba48-4383-8d6e-9ae67002fa68 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de934d0105bd4d01ba6c06a4a0d36704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:43.524563 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8652b1-ba48-4383-8d6e-9ae67002fa68 service placement] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:43.525128 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8652b1-ba48-4383-8d6e-9ae67002fa68 service placement] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:43.525971 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8652b1-ba48-4383-8d6e-9ae67002fa68 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:43.547260 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8652b1-ba48-4383-8d6e-9ae67002fa68 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:43.550576 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8652b1-ba48-4383-8d6e-9ae67002fa68 service placement] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:43.583093 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-132c7ea2-eff0-42c4-a0fb-80abd35fec77 None None] MFA Rules not processed for user `298dfca23e004969a0c907e18c7237a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:43.584305 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-132c7ea2-eff0-42c4-a0fb-80abd35fec77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:43.590659 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 846/1677] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:41:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:41:43.594482 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1236c732-a443-4607-b678-8d3cb736bf3f None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:43.596685 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1236c732-a443-4607-b678-8d3cb736bf3f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:43.597511 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1236c732-a443-4607-b678-8d3cb736bf3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:43.659708 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-1236c732-a443-4607-b678-8d3cb736bf3f None None] MFA Rules not processed for user `36515ae6f91748babdcf6a9a72f59ed1`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:43.661705 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1236c732-a443-4607-b678-8d3cb736bf3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:43.751655 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 832/1678] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:43 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:41:43.753024 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2bf9c-8fa6-4ce5-8ff4-b87b847584e1 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:43.753471 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2bf9c-8fa6-4ce5-8ff4-b87b847584e1 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:43.753577 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2bf9c-8fa6-4ce5-8ff4-b87b847584e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:43.773546 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 847/1679] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:43 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:43.780798 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1236c732-a443-4607-b678-8d3cb736bf3f None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:43.782997 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1236c732-a443-4607-b678-8d3cb736bf3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:43.828255 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-7ed2bf9c-8fa6-4ce5-8ff4-b87b847584e1 None None] MFA Rules not processed for user `922e0b1b4717422abb6f4e93ea15a86d`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:43.832160 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7ed2bf9c-8fa6-4ce5-8ff4-b87b847584e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:43.840687 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:41:43.863373 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:41:43.905995 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1236c732-a443-4607-b678-8d3cb736bf3f None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:43.914509 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9b851b-33e5-453c-b9fd-d72c44ffd617 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf14b64bf04b2ba74d2fe1fd41ff71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:43.915312 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b851b-33e5-453c-b9fd-d72c44ffd617 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:43.915684 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b851b-33e5-453c-b9fd-d72c44ffd617 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:43.916051 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b851b-33e5-453c-b9fd-d72c44ffd617 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:43.929708 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9b851b-33e5-453c-b9fd-d72c44ffd617 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:43.937660 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9b851b-33e5-453c-b9fd-d72c44ffd617 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:43.951763 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 848/1680] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:41:43 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:41:43.953120 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a06630-365e-4d7c-987c-52f3bf0f1f1c None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:43.953507 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a06630-365e-4d7c-987c-52f3bf0f1f1c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:43.954988 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a06630-365e-4d7c-987c-52f3bf0f1f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:43.980043 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 833/1681] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:43.983808 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08e24b-2553-4439-bb06-d59a0320620a None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:43.984110 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08e24b-2553-4439-bb06-d59a0320620a None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:43.984366 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08e24b-2553-4439-bb06-d59a0320620a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:43.997082 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-f3a06630-365e-4d7c-987c-52f3bf0f1f1c None None] MFA Rules not processed for user `36515ae6f91748babdcf6a9a72f59ed1`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:43.998706 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f3a06630-365e-4d7c-987c-52f3bf0f1f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:44.014712 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 849/1682] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:43 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:44.026495 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a06630-365e-4d7c-987c-52f3bf0f1f1c None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:44.034097 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-af08e24b-2553-4439-bb06-d59a0320620a None None] MFA Rules not processed for user `922e0b1b4717422abb6f4e93ea15a86d`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:44.035329 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-af08e24b-2553-4439-bb06-d59a0320620a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:44.036072 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f3a06630-365e-4d7c-987c-52f3bf0f1f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:44.064429 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 834/1683] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:41:44.081229 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a06630-365e-4d7c-987c-52f3bf0f1f1c None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:44.095739 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56148e73-89af-4423-92ef-0f64ba9489d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf14b64bf04b2ba74d2fe1fd41ff71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:44.097685 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56148e73-89af-4423-92ef-0f64ba9489d8 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:44.099685 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56148e73-89af-4423-92ef-0f64ba9489d8 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:44.100558 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56148e73-89af-4423-92ef-0f64ba9489d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:44.129783 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af08e24b-2553-4439-bb06-d59a0320620a None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:44.157658 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af08e24b-2553-4439-bb06-d59a0320620a None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:44.158412 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56148e73-89af-4423-92ef-0f64ba9489d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:44.161274 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56148e73-89af-4423-92ef-0f64ba9489d8 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:44.170804 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763b9b4-0848-4402-9771-01ef696fa244 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf14b64bf04b2ba74d2fe1fd41ff71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:44.171702 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763b9b4-0848-4402-9771-01ef696fa244 service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:44.172128 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763b9b4-0848-4402-9771-01ef696fa244 service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:44.172527 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763b9b4-0848-4402-9771-01ef696fa244 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:44.179624 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 850/1684] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:41:44 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:41:44.202806 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a763b9b4-0848-4402-9771-01ef696fa244 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:44.204425 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a763b9b4-0848-4402-9771-01ef696fa244 service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:44.256535 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 835/1685] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:41:44 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:41:45.267962 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56148e73-89af-4423-92ef-0f64ba9489d8 service nova] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:45.269999 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-56148e73-89af-4423-92ef-0f64ba9489d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:45.299851 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56148e73-89af-4423-92ef-0f64ba9489d8 service nova] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:45.320385 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32e2981-5fc7-42f6-97f3-38352d8ced56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:45.321514 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32e2981-5fc7-42f6-97f3-38352d8ced56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:45.322373 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32e2981-5fc7-42f6-97f3-38352d8ced56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:45.323552 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32e2981-5fc7-42f6-97f3-38352d8ced56 admin admin] PATH_INFO: `/v3/projects/e4c3c0db462e41488735ffe3589beb8a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:45.326688 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32e2981-5fc7-42f6-97f3-38352d8ced56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4c3c0db462e41488735ffe3589beb8a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:45.335218 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: WARNING py.warnings [None req-d32e2981-5fc7-42f6-97f3-38352d8ced56 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 28 15:41:45.335218 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: warnings.warn(msg) Sep 28 15:41:45.335218 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: Sep 28 15:41:45.337320 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32e2981-5fc7-42f6-97f3-38352d8ced56 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:45.464417 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d32e2981-5fc7-42f6-97f3-38352d8ced56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4c3c0db462e41488735ffe3589beb8a', 'request_id': 'req-d32e2981-5fc7-42f6-97f3-38352d8ced56'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:45.497377 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d32e2981-5fc7-42f6-97f3-38352d8ced56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4c3c0db462e41488735ffe3589beb8a', 'request_id': 'req-d32e2981-5fc7-42f6-97f3-38352d8ced56'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:45.516035 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d32e2981-5fc7-42f6-97f3-38352d8ced56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4c3c0db462e41488735ffe3589beb8a', 'request_id': 'req-d32e2981-5fc7-42f6-97f3-38352d8ced56'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:45.588768 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d32e2981-5fc7-42f6-97f3-38352d8ced56 admin admin] The token cache is being invalidate because project e4c3c0db462e41488735ffe3589beb8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:41:45.589373 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d32e2981-5fc7-42f6-97f3-38352d8ced56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:45.591514 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d32e2981-5fc7-42f6-97f3-38352d8ced56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:45.602991 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 851/1686] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:41:45 2022] DELETE /identity/v3/projects/e4c3c0db462e41488735ffe3589beb8a => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:41:45.607261 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763b9b4-0848-4402-9771-01ef696fa244 service nova] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:45.620317 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763b9b4-0848-4402-9771-01ef696fa244 service nova] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:45.748333 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801ba295-54f8-4243-b7c9-e08dc969d549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:45.749371 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ba295-54f8-4243-b7c9-e08dc969d549 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:45.749931 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ba295-54f8-4243-b7c9-e08dc969d549 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:45.750357 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ba295-54f8-4243-b7c9-e08dc969d549 admin admin] PATH_INFO: `/v3/users/7a31c2863ca54217a709a0227495f7c9` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:45.751961 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801ba295-54f8-4243-b7c9-e08dc969d549 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a31c2863ca54217a709a0227495f7c9)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:45.755873 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801ba295-54f8-4243-b7c9-e08dc969d549 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:45.822723 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32e2981-5fc7-42f6-97f3-38352d8ced56 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:45.824941 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d32e2981-5fc7-42f6-97f3-38352d8ced56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:45.842121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32e2981-5fc7-42f6-97f3-38352d8ced56 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:45.851540 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72456e78-cf77-4bcb-ab1f-08c5f9e50b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:45.852413 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72456e78-cf77-4bcb-ab1f-08c5f9e50b00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:45.853590 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72456e78-cf77-4bcb-ab1f-08c5f9e50b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:45.854058 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72456e78-cf77-4bcb-ab1f-08c5f9e50b00 admin admin] PATH_INFO: `/v3/users/5108b5cf3d3948379ccefa22fe573784` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:45.868770 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72456e78-cf77-4bcb-ab1f-08c5f9e50b00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5108b5cf3d3948379ccefa22fe573784)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:45.870456 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-801ba295-54f8-4243-b7c9-e08dc969d549 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a31c2863ca54217a709a0227495f7c9'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:45.882302 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72456e78-cf77-4bcb-ab1f-08c5f9e50b00 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:45.882633 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-801ba295-54f8-4243-b7c9-e08dc969d549 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a31c2863ca54217a709a0227495f7c9'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:45.883502 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-801ba295-54f8-4243-b7c9-e08dc969d549 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a31c2863ca54217a709a0227495f7c9'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:45.910094 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-801ba295-54f8-4243-b7c9-e08dc969d549 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a31c2863ca54217a709a0227495f7c9'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:45.911566 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-801ba295-54f8-4243-b7c9-e08dc969d549 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a31c2863ca54217a709a0227495f7c9'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:45.946116 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 836/1687] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:41:45 2022] DELETE /identity/v3/users/7a31c2863ca54217a709a0227495f7c9 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:41:45.947489 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf815e-4cf4-4798-991a-8fe682e03cc6 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:45.947774 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf815e-4cf4-4798-991a-8fe682e03cc6 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:45.948084 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faf815e-4cf4-4798-991a-8fe682e03cc6 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:45.950740 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 837/1688] 10.222.0.52 () {54 vars in 912 bytes} [Wed Sep 28 15:41:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:41:45.953244 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0cad90-1048-4c51-821c-fce380d6ca9e None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:45.953651 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0cad90-1048-4c51-821c-fce380d6ca9e None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:45.953956 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0cad90-1048-4c51-821c-fce380d6ca9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:46.002054 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-5e0cad90-1048-4c51-821c-fce380d6ca9e None None] MFA Rules not processed for user `f22b26ed7a47480e8448175517170172`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:46.006421 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-5e0cad90-1048-4c51-821c-fce380d6ca9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:46.021734 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-72456e78-cf77-4bcb-ab1f-08c5f9e50b00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5108b5cf3d3948379ccefa22fe573784'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:46.023513 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-72456e78-cf77-4bcb-ab1f-08c5f9e50b00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5108b5cf3d3948379ccefa22fe573784'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:46.032874 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:41:46.051446 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-72456e78-cf77-4bcb-ab1f-08c5f9e50b00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5108b5cf3d3948379ccefa22fe573784'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:46.075972 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-72456e78-cf77-4bcb-ab1f-08c5f9e50b00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5108b5cf3d3948379ccefa22fe573784'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:46.079177 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:41:46.089514 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-72456e78-cf77-4bcb-ab1f-08c5f9e50b00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5108b5cf3d3948379ccefa22fe573784'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:46.106635 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 852/1689] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:41:45 2022] DELETE /identity/v3/users/5108b5cf3d3948379ccefa22fe573784 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:41:46.108189 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0057cd47-d25c-4d00-9969-2b2588df56a5 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:46.108584 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0057cd47-d25c-4d00-9969-2b2588df56a5 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:46.108963 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0057cd47-d25c-4d00-9969-2b2588df56a5 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:46.116351 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 853/1690] 10.222.0.52 () {54 vars in 912 bytes} [Wed Sep 28 15:41:46 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:41:46.117561 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb72037-0a56-4b7a-a2f9-63bcf6f65a0b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:46.118000 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb72037-0a56-4b7a-a2f9-63bcf6f65a0b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:46.118658 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb72037-0a56-4b7a-a2f9-63bcf6f65a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:46.181406 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 838/1691] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:46.182840 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc62d65-84d7-4709-9ff2-c0bfe2fa5ffb None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:46.183123 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc62d65-84d7-4709-9ff2-c0bfe2fa5ffb None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:46.183405 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc62d65-84d7-4709-9ff2-c0bfe2fa5ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:46.200648 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-dfb72037-0a56-4b7a-a2f9-63bcf6f65a0b None None] MFA Rules not processed for user `b13b0533444a449ead00a389c5f6bd1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:46.203626 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-dfb72037-0a56-4b7a-a2f9-63bcf6f65a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:46.224021 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-5cc62d65-84d7-4709-9ff2-c0bfe2fa5ffb None None] MFA Rules not processed for user `b13b0533444a449ead00a389c5f6bd1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:46.227720 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-5cc62d65-84d7-4709-9ff2-c0bfe2fa5ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:46.352015 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 854/1692] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Sep 28 15:41:46 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:46.366969 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 839/1693] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Sep 28 15:41:46 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:46.425892 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb72037-0a56-4b7a-a2f9-63bcf6f65a0b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:46.428138 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc62d65-84d7-4709-9ff2-c0bfe2fa5ffb None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:46.428782 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-dfb72037-0a56-4b7a-a2f9-63bcf6f65a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:46.429872 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-5cc62d65-84d7-4709-9ff2-c0bfe2fa5ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:46.449316 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc62d65-84d7-4709-9ff2-c0bfe2fa5ffb None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:46.455189 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb72037-0a56-4b7a-a2f9-63bcf6f65a0b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:46.458463 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3efcce3-d53d-4443-9d6c-619fdf68cd67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:46.459816 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3efcce3-d53d-4443-9d6c-619fdf68cd67 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:46.460236 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3efcce3-d53d-4443-9d6c-619fdf68cd67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:46.462412 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3efcce3-d53d-4443-9d6c-619fdf68cd67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:46.466608 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65813c6e-9011-4f38-9af7-a2c213fde6d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:46.468663 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65813c6e-9011-4f38-9af7-a2c213fde6d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:46.469356 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65813c6e-9011-4f38-9af7-a2c213fde6d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:46.471031 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65813c6e-9011-4f38-9af7-a2c213fde6d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:46.486808 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3efcce3-d53d-4443-9d6c-619fdf68cd67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:46.488536 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3efcce3-d53d-4443-9d6c-619fdf68cd67 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:46.490574 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65813c6e-9011-4f38-9af7-a2c213fde6d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:46.493719 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65813c6e-9011-4f38-9af7-a2c213fde6d8 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:46.509961 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 855/1694] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:41:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:41:46.514696 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 840/1695] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:41:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:41:47.070223 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65813c6e-9011-4f38-9af7-a2c213fde6d8 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:47.071907 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-65813c6e-9011-4f38-9af7-a2c213fde6d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:47.191893 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65813c6e-9011-4f38-9af7-a2c213fde6d8 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:47.196715 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a17cd-1b4e-4e34-9229-279c49e1ca73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:47.197360 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a17cd-1b4e-4e34-9229-279c49e1ca73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:47.197641 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a17cd-1b4e-4e34-9229-279c49e1ca73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:47.198029 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a17cd-1b4e-4e34-9229-279c49e1ca73 admin admin] PATH_INFO: `/v3/projects/d7287e9f86c64792bcdf362bba013439` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:47.199845 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8a17cd-1b4e-4e34-9229-279c49e1ca73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7287e9f86c64792bcdf362bba013439)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:47.201979 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8a17cd-1b4e-4e34-9229-279c49e1ca73 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:47.284167 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4f8a17cd-1b4e-4e34-9229-279c49e1ca73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7287e9f86c64792bcdf362bba013439', 'request_id': 'req-4f8a17cd-1b4e-4e34-9229-279c49e1ca73'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:47.295400 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3efcce3-d53d-4443-9d6c-619fdf68cd67 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:47.298584 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c3efcce3-d53d-4443-9d6c-619fdf68cd67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:47.301498 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4f8a17cd-1b4e-4e34-9229-279c49e1ca73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7287e9f86c64792bcdf362bba013439', 'request_id': 'req-4f8a17cd-1b4e-4e34-9229-279c49e1ca73'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:47.317087 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4f8a17cd-1b4e-4e34-9229-279c49e1ca73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7287e9f86c64792bcdf362bba013439', 'request_id': 'req-4f8a17cd-1b4e-4e34-9229-279c49e1ca73'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:47.325348 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3efcce3-d53d-4443-9d6c-619fdf68cd67 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:47.345772 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4f8a17cd-1b4e-4e34-9229-279c49e1ca73 admin admin] The token cache is being invalidate because project d7287e9f86c64792bcdf362bba013439 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:41:47.348354 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4f8a17cd-1b4e-4e34-9229-279c49e1ca73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:47.351063 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4f8a17cd-1b4e-4e34-9229-279c49e1ca73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:47.355201 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 856/1696] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:41:47 2022] DELETE /identity/v3/projects/d7287e9f86c64792bcdf362bba013439 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:41:47.360894 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a17cd-1b4e-4e34-9229-279c49e1ca73 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:47.362923 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4f8a17cd-1b4e-4e34-9229-279c49e1ca73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:47.380802 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:41:47.506909 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d07e54c-5bff-41dd-b9a9-972701fdfb29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:47.507809 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d07e54c-5bff-41dd-b9a9-972701fdfb29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:47.508638 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d07e54c-5bff-41dd-b9a9-972701fdfb29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:47.511028 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d07e54c-5bff-41dd-b9a9-972701fdfb29 admin admin] PATH_INFO: `/v3/projects/4f0eaf5519ec4454a3e6f6943c0d8486` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:47.514395 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d07e54c-5bff-41dd-b9a9-972701fdfb29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f0eaf5519ec4454a3e6f6943c0d8486)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:47.517422 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d07e54c-5bff-41dd-b9a9-972701fdfb29 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:47.561287 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a17cd-1b4e-4e34-9229-279c49e1ca73 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:47.580557 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f45f4-785a-4b10-bbcc-3b7615bd2333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:47.583281 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f45f4-785a-4b10-bbcc-3b7615bd2333 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:47.583519 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f45f4-785a-4b10-bbcc-3b7615bd2333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:47.583799 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f45f4-785a-4b10-bbcc-3b7615bd2333 admin admin] PATH_INFO: `/v3/users/715c52a2ce0742d19a9c5b04cdd536e1` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:47.586704 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3f45f4-785a-4b10-bbcc-3b7615bd2333 admin admin] RBAC: Authorizing `identity:delete_user(user_id=715c52a2ce0742d19a9c5b04cdd536e1)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:47.591278 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3f45f4-785a-4b10-bbcc-3b7615bd2333 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:47.639236 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3d07e54c-5bff-41dd-b9a9-972701fdfb29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f0eaf5519ec4454a3e6f6943c0d8486', 'request_id': 'req-3d07e54c-5bff-41dd-b9a9-972701fdfb29'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:47.660195 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3d07e54c-5bff-41dd-b9a9-972701fdfb29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f0eaf5519ec4454a3e6f6943c0d8486', 'request_id': 'req-3d07e54c-5bff-41dd-b9a9-972701fdfb29'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:47.664040 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:41:47.670003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3d07e54c-5bff-41dd-b9a9-972701fdfb29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f0eaf5519ec4454a3e6f6943c0d8486', 'request_id': 'req-3d07e54c-5bff-41dd-b9a9-972701fdfb29'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:47.694866 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3d07e54c-5bff-41dd-b9a9-972701fdfb29 admin admin] The token cache is being invalidate because project 4f0eaf5519ec4454a3e6f6943c0d8486 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:41:47.696570 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3d07e54c-5bff-41dd-b9a9-972701fdfb29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:47.698104 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3d07e54c-5bff-41dd-b9a9-972701fdfb29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:47.703249 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 841/1697] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:41:47 2022] DELETE /identity/v3/projects/4f0eaf5519ec4454a3e6f6943c0d8486 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:41:47.711435 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf7177-0224-4cfd-a0c1-b3c344ce2d6e None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:47.711722 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf7177-0224-4cfd-a0c1-b3c344ce2d6e None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:47.711992 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf7177-0224-4cfd-a0c1-b3c344ce2d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:47.713191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1e3f45f4-785a-4b10-bbcc-3b7615bd2333 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '715c52a2ce0742d19a9c5b04cdd536e1'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:47.714144 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1e3f45f4-785a-4b10-bbcc-3b7615bd2333 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '715c52a2ce0742d19a9c5b04cdd536e1'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:47.743437 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1e3f45f4-785a-4b10-bbcc-3b7615bd2333 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '715c52a2ce0742d19a9c5b04cdd536e1'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:47.758210 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-1fbf7177-0224-4cfd-a0c1-b3c344ce2d6e None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:47.759914 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-1fbf7177-0224-4cfd-a0c1-b3c344ce2d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:47.766345 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1e3f45f4-785a-4b10-bbcc-3b7615bd2333 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '715c52a2ce0742d19a9c5b04cdd536e1'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:47.778444 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1e3f45f4-785a-4b10-bbcc-3b7615bd2333 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '715c52a2ce0742d19a9c5b04cdd536e1'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:47.788721 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 857/1698] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:41:47 2022] DELETE /identity/v3/users/715c52a2ce0742d19a9c5b04cdd536e1 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:41:47.931965 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 842/1699] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:47.939893 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322843a-255b-4100-8e13-3a7efd1a8e36 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:47.940498 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322843a-255b-4100-8e13-3a7efd1a8e36 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:47.940935 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322843a-255b-4100-8e13-3a7efd1a8e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:48.014613 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-4322843a-255b-4100-8e13-3a7efd1a8e36 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:48.021239 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4322843a-255b-4100-8e13-3a7efd1a8e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:48.221268 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 858/1700] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:48.223891 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbf7177-0224-4cfd-a0c1-b3c344ce2d6e None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:48.240731 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbf7177-0224-4cfd-a0c1-b3c344ce2d6e None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:48.270606 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02015f6d-75e6-493e-9a6e-7be8d58b2009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:48.271244 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02015f6d-75e6-493e-9a6e-7be8d58b2009 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:48.271523 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02015f6d-75e6-493e-9a6e-7be8d58b2009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:48.271794 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02015f6d-75e6-493e-9a6e-7be8d58b2009 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:48.278516 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02015f6d-75e6-493e-9a6e-7be8d58b2009 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:41:48.278909 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02015f6d-75e6-493e-9a6e-7be8d58b2009 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:48.281118 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02015f6d-75e6-493e-9a6e-7be8d58b2009 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:48.300678 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 843/1701] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:41:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:41:48.305074 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322843a-255b-4100-8e13-3a7efd1a8e36 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:48.317999 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322843a-255b-4100-8e13-3a7efd1a8e36 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:48.331099 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e8acec-df73-499d-8c3a-d732e6a1a891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:48.332264 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e8acec-df73-499d-8c3a-d732e6a1a891 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:48.332400 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e8acec-df73-499d-8c3a-d732e6a1a891 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:48.332559 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e8acec-df73-499d-8c3a-d732e6a1a891 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:48.333200 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e8acec-df73-499d-8c3a-d732e6a1a891 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:48.335246 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e8acec-df73-499d-8c3a-d732e6a1a891 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:48.399734 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 859/1702] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:41:48 2022] POST /identity/v3/projects => generated 365 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:41:48.404298 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02015f6d-75e6-493e-9a6e-7be8d58b2009 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:48.428575 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02015f6d-75e6-493e-9a6e-7be8d58b2009 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:48.650276 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bc3996-7770-4225-b21a-2917150e0b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:48.651108 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc3996-7770-4225-b21a-2917150e0b2b admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:48.651486 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc3996-7770-4225-b21a-2917150e0b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:48.651819 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc3996-7770-4225-b21a-2917150e0b2b admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:48.653939 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bc3996-7770-4225-b21a-2917150e0b2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:48.656234 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bc3996-7770-4225-b21a-2917150e0b2b admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:48.698908 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e8acec-df73-499d-8c3a-d732e6a1a891 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:48.701512 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-89e8acec-df73-499d-8c3a-d732e6a1a891 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:48.719705 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:41:48.728262 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 844/1703] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:41:48 2022] POST /identity/v3/users => generated 283 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:41:48.729472 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bc3996-7770-4225-b21a-2917150e0b2b admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:48.751836 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bc3996-7770-4225-b21a-2917150e0b2b admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:48.795030 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fda3ab-d2c3-43ac-bfbe-5cf70a94892d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:48.795651 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fda3ab-d2c3-43ac-bfbe-5cf70a94892d admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:48.795927 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fda3ab-d2c3-43ac-bfbe-5cf70a94892d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:48.796197 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fda3ab-d2c3-43ac-bfbe-5cf70a94892d admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:48.796845 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fda3ab-d2c3-43ac-bfbe-5cf70a94892d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:48.799256 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fda3ab-d2c3-43ac-bfbe-5cf70a94892d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:48.842651 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 845/1704] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:41:48 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:41:48.847627 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fda3ab-d2c3-43ac-bfbe-5cf70a94892d admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:48.904296 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fda3ab-d2c3-43ac-bfbe-5cf70a94892d admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:48.924132 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994717ab-418f-4b39-9d7e-522bc92b4a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:48.924774 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994717ab-418f-4b39-9d7e-522bc92b4a14 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:48.925053 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994717ab-418f-4b39-9d7e-522bc92b4a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:48.925324 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994717ab-418f-4b39-9d7e-522bc92b4a14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:48.926016 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994717ab-418f-4b39-9d7e-522bc92b4a14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:48.937943 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994717ab-418f-4b39-9d7e-522bc92b4a14 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:48.980582 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 846/1705] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:41:48 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:41:48.986950 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994717ab-418f-4b39-9d7e-522bc92b4a14 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:49.033341 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994717ab-418f-4b39-9d7e-522bc92b4a14 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:49.039910 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e8acec-df73-499d-8c3a-d732e6a1a891 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:49.045140 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb477e45-80b3-4586-bec3-6771e17ef61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:49.046100 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb477e45-80b3-4586-bec3-6771e17ef61a admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:49.046410 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb477e45-80b3-4586-bec3-6771e17ef61a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:49.046838 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb477e45-80b3-4586-bec3-6771e17ef61a admin admin] PATH_INFO: `/v3/projects/3493d310d37d4ff6a2525b58f9fe9609/users/e42b0ef560e14681876d635648cc01fc/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:49.048211 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889ea1a7-69f9-486d-b782-d5fccc45f07d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de934d0105bd4d01ba6c06a4a0d36704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:49.049926 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889ea1a7-69f9-486d-b782-d5fccc45f07d service placement] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:49.050425 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889ea1a7-69f9-486d-b782-d5fccc45f07d service placement] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:49.051977 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889ea1a7-69f9-486d-b782-d5fccc45f07d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:49.054271 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-889ea1a7-69f9-486d-b782-d5fccc45f07d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:49.078679 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889ea1a7-69f9-486d-b782-d5fccc45f07d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:49.080313 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889ea1a7-69f9-486d-b782-d5fccc45f07d service placement] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:49.098989 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb477e45-80b3-4586-bec3-6771e17ef61a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3493d310d37d4ff6a2525b58f9fe9609, user_id=e42b0ef560e14681876d635648cc01fc, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:49.104236 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb477e45-80b3-4586-bec3-6771e17ef61a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:49.107115 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 860/1706] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:41:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:41:49.108487 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa33e8-b588-478a-9382-742fcb3e2b14 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:49.108778 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa33e8-b588-478a-9382-742fcb3e2b14 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:49.109052 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaa33e8-b588-478a-9382-742fcb3e2b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:49.156056 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 847/1707] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:41:48 2022] PUT /identity/v3/projects/3493d310d37d4ff6a2525b58f9fe9609/users/e42b0ef560e14681876d635648cc01fc/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:41:49.158423 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe8fb5-4336-4a0d-85fd-708591a6b8cd None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:49.158974 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe8fb5-4336-4a0d-85fd-708591a6b8cd None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:49.159446 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe8fb5-4336-4a0d-85fd-708591a6b8cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:49.181304 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-aeaa33e8-b588-478a-9382-742fcb3e2b14 None None] MFA Rules not processed for user `298dfca23e004969a0c907e18c7237a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:49.183259 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-aeaa33e8-b588-478a-9382-742fcb3e2b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:49.266712 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-defe8fb5-4336-4a0d-85fd-708591a6b8cd None None] MFA Rules not processed for user `e42b0ef560e14681876d635648cc01fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:49.269819 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-defe8fb5-4336-4a0d-85fd-708591a6b8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:49.300657 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:41:49.317972 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 861/1708] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:49 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:49.319080 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaa33e8-b588-478a-9382-742fcb3e2b14 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:49.320733 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-aeaa33e8-b588-478a-9382-742fcb3e2b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:49.440583 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 848/1709] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:49.442383 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defe8fb5-4336-4a0d-85fd-708591a6b8cd None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:49.449837 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaa33e8-b588-478a-9382-742fcb3e2b14 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:49.456068 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defe8fb5-4336-4a0d-85fd-708591a6b8cd None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:49.457508 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b7b001-cedc-42c1-81a3-e8a37e4a7a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:49.458522 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b7b001-cedc-42c1-81a3-e8a37e4a7a5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:49.459049 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b7b001-cedc-42c1-81a3-e8a37e4a7a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:49.459495 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b7b001-cedc-42c1-81a3-e8a37e4a7a5e admin admin] PATH_INFO: `/v3/projects/fd72bda4e1944528a2f342e3a25173d0` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:49.461098 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6da0d92-7ea4-4241-b5a0-34ec241083ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:49.461838 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da0d92-7ea4-4241-b5a0-34ec241083ca admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:49.462249 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b7b001-cedc-42c1-81a3-e8a37e4a7a5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd72bda4e1944528a2f342e3a25173d0)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:49.462363 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da0d92-7ea4-4241-b5a0-34ec241083ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:49.463151 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da0d92-7ea4-4241-b5a0-34ec241083ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:49.463789 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6da0d92-7ea4-4241-b5a0-34ec241083ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:49.464936 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b7b001-cedc-42c1-81a3-e8a37e4a7a5e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:49.467025 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6da0d92-7ea4-4241-b5a0-34ec241083ca admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:49.545695 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:41:49.548604 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 849/1710] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:41:49 2022] POST /identity/v3/projects => generated 363 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:41:49.553256 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6da0d92-7ea4-4241-b5a0-34ec241083ca admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:49.560438 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-07b7b001-cedc-42c1-81a3-e8a37e4a7a5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd72bda4e1944528a2f342e3a25173d0', 'request_id': 'req-07b7b001-cedc-42c1-81a3-e8a37e4a7a5e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:49.569618 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-07b7b001-cedc-42c1-81a3-e8a37e4a7a5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd72bda4e1944528a2f342e3a25173d0', 'request_id': 'req-07b7b001-cedc-42c1-81a3-e8a37e4a7a5e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:49.580344 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-07b7b001-cedc-42c1-81a3-e8a37e4a7a5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd72bda4e1944528a2f342e3a25173d0', 'request_id': 'req-07b7b001-cedc-42c1-81a3-e8a37e4a7a5e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:49.609332 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-07b7b001-cedc-42c1-81a3-e8a37e4a7a5e admin admin] The token cache is being invalidate because project fd72bda4e1944528a2f342e3a25173d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:41:49.609950 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-07b7b001-cedc-42c1-81a3-e8a37e4a7a5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:49.610881 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-07b7b001-cedc-42c1-81a3-e8a37e4a7a5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:49.612748 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 862/1711] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:41:49 2022] DELETE /identity/v3/projects/fd72bda4e1944528a2f342e3a25173d0 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:41:49.618482 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf0c70-b9df-45d9-825d-95189d19c80d None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:49.619181 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf0c70-b9df-45d9-825d-95189d19c80d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:49.619658 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf0c70-b9df-45d9-825d-95189d19c80d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:49.667405 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-9aaf0c70-b9df-45d9-825d-95189d19c80d None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:49.671318 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6da0d92-7ea4-4241-b5a0-34ec241083ca admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:49.672362 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9aaf0c70-b9df-45d9-825d-95189d19c80d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:49.687219 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441f5f05-dfc6-413f-afe6-4aa1909452ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:49.688224 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f5f05-dfc6-413f-afe6-4aa1909452ab admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:49.688517 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f5f05-dfc6-413f-afe6-4aa1909452ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:49.688825 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f5f05-dfc6-413f-afe6-4aa1909452ab admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:49.689590 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441f5f05-dfc6-413f-afe6-4aa1909452ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:49.692083 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441f5f05-dfc6-413f-afe6-4aa1909452ab admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:49.701070 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 863/1712] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:41:49.702597 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe27fb8-0cad-44b8-a6ec-41d89e837f80 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:49.703233 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe27fb8-0cad-44b8-a6ec-41d89e837f80 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:49.706286 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe27fb8-0cad-44b8-a6ec-41d89e837f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:49.762409 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 850/1713] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:41:49 2022] POST /identity/v3/users => generated 288 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:41:49.764187 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b952d-61e1-4615-9a09-563ce27ffa87 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:49.764469 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b952d-61e1-4615-9a09-563ce27ffa87 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:49.764742 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b952d-61e1-4615-9a09-563ce27ffa87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:49.770823 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-1fe27fb8-0cad-44b8-a6ec-41d89e837f80 None None] MFA Rules not processed for user `634c0af05b334b488f2fa358e574172b`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:49.772692 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1fe27fb8-0cad-44b8-a6ec-41d89e837f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:49.813374 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-841b952d-61e1-4615-9a09-563ce27ffa87 None None] MFA Rules not processed for user `36515ae6f91748babdcf6a9a72f59ed1`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:49.813374 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-841b952d-61e1-4615-9a09-563ce27ffa87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:49.961006 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 864/1714] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:49.969211 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00046d4-86f8-4314-805b-dcd7102dac82 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:49.969509 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00046d4-86f8-4314-805b-dcd7102dac82 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:49.969786 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00046d4-86f8-4314-805b-dcd7102dac82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:49.973740 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 851/1715] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:49 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:49.974972 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841b952d-61e1-4615-9a09-563ce27ffa87 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:49.978516 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-841b952d-61e1-4615-9a09-563ce27ffa87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:49.993953 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-e00046d4-86f8-4314-805b-dcd7102dac82 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:49.996000 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e00046d4-86f8-4314-805b-dcd7102dac82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:50.013356 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 865/1716] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:50.014700 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00046d4-86f8-4314-805b-dcd7102dac82 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:50.031310 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00046d4-86f8-4314-805b-dcd7102dac82 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:50.037251 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd614d-c624-4695-9c0a-91884ae70227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:50.037896 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd614d-c624-4695-9c0a-91884ae70227 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:50.038582 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd614d-c624-4695-9c0a-91884ae70227 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:50.039032 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd614d-c624-4695-9c0a-91884ae70227 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:50.040066 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccd614d-c624-4695-9c0a-91884ae70227 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:41:50.040377 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccd614d-c624-4695-9c0a-91884ae70227 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:50.042226 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccd614d-c624-4695-9c0a-91884ae70227 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:50.045329 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841b952d-61e1-4615-9a09-563ce27ffa87 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:50.088375 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1374ca2e-bcd1-4fda-ab37-c980208a6099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:50.090985 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1374ca2e-bcd1-4fda-ab37-c980208a6099 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:50.093621 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1374ca2e-bcd1-4fda-ab37-c980208a6099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:50.100712 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1374ca2e-bcd1-4fda-ab37-c980208a6099 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:50.105011 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1374ca2e-bcd1-4fda-ab37-c980208a6099 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:50.106400 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 866/1717] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:41:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:41:50.108422 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1374ca2e-bcd1-4fda-ab37-c980208a6099 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:50.111231 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd614d-c624-4695-9c0a-91884ae70227 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:50.157304 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd614d-c624-4695-9c0a-91884ae70227 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:50.175658 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 852/1718] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:41:49 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:41:50.175816 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfe1c1f-96a3-4d3a-84bb-0a15d4ada9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:50.178208 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe1c1f-96a3-4d3a-84bb-0a15d4ada9ab admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:50.179167 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe1c1f-96a3-4d3a-84bb-0a15d4ada9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:50.180250 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe1c1f-96a3-4d3a-84bb-0a15d4ada9ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:50.183989 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfe1c1f-96a3-4d3a-84bb-0a15d4ada9ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:50.188904 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1374ca2e-bcd1-4fda-ab37-c980208a6099 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:50.192044 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfe1c1f-96a3-4d3a-84bb-0a15d4ada9ab admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:50.219395 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1374ca2e-bcd1-4fda-ab37-c980208a6099 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:50.231661 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbcfbd6-5870-490c-ba4b-2ab812fc16b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:50.234682 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbcfbd6-5870-490c-ba4b-2ab812fc16b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:50.235026 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbcfbd6-5870-490c-ba4b-2ab812fc16b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:50.235299 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbcfbd6-5870-490c-ba4b-2ab812fc16b3 admin admin] PATH_INFO: `/v3/projects/c61f19562a264e36b2c6f2df0d1c8013/users/ef12528aa8d34059887a81df61e47833/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:50.243077 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:41:50.279659 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 867/1719] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:41:50 2022] POST /identity/v3/projects => generated 385 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:41:50.289005 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbcfbd6-5870-490c-ba4b-2ab812fc16b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c61f19562a264e36b2c6f2df0d1c8013, user_id=ef12528aa8d34059887a81df61e47833, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:50.291973 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbcfbd6-5870-490c-ba4b-2ab812fc16b3 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:50.303047 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfe1c1f-96a3-4d3a-84bb-0a15d4ada9ab admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:50.348787 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 853/1720] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:41:50 2022] PUT /identity/v3/projects/c61f19562a264e36b2c6f2df0d1c8013/users/ef12528aa8d34059887a81df61e47833/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:41:50.350143 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbcfbd6-5870-490c-ba4b-2ab812fc16b3 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:50.477366 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbcfbd6-5870-490c-ba4b-2ab812fc16b3 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:50.485041 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cac18e0-cb42-4df1-a6c3-ac38d8059afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:50.486981 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac18e0-cb42-4df1-a6c3-ac38d8059afe admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:50.487380 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac18e0-cb42-4df1-a6c3-ac38d8059afe admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:50.487749 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac18e0-cb42-4df1-a6c3-ac38d8059afe admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:50.488725 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cac18e0-cb42-4df1-a6c3-ac38d8059afe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:50.492584 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cac18e0-cb42-4df1-a6c3-ac38d8059afe admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:50.498087 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfe1c1f-96a3-4d3a-84bb-0a15d4ada9ab admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:50.509059 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 854/1721] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:41:50 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:41:50.514865 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cac18e0-cb42-4df1-a6c3-ac38d8059afe admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:50.519897 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b927451-84a0-45f0-9ad5-337ba3c8ab14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:50.520572 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b927451-84a0-45f0-9ad5-337ba3c8ab14 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:50.520829 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b927451-84a0-45f0-9ad5-337ba3c8ab14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:50.521292 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b927451-84a0-45f0-9ad5-337ba3c8ab14 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:50.521966 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b927451-84a0-45f0-9ad5-337ba3c8ab14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:50.523901 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b927451-84a0-45f0-9ad5-337ba3c8ab14 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:50.540878 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cac18e0-cb42-4df1-a6c3-ac38d8059afe admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:50.546774 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8456cee4-faf6-474b-a32e-30cf57357ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:50.547572 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8456cee4-faf6-474b-a32e-30cf57357ca2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:50.548091 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8456cee4-faf6-474b-a32e-30cf57357ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:50.548460 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8456cee4-faf6-474b-a32e-30cf57357ca2 admin admin] PATH_INFO: `/v3/projects/c61f19562a264e36b2c6f2df0d1c8013/users/ef12528aa8d34059887a81df61e47833/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:50.552149 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8456cee4-faf6-474b-a32e-30cf57357ca2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c61f19562a264e36b2c6f2df0d1c8013, user_id=ef12528aa8d34059887a81df61e47833, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:50.556371 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8456cee4-faf6-474b-a32e-30cf57357ca2 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:50.571059 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 868/1722] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:41:50 2022] POST /identity/v3/users => generated 293 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:41:50.577028 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b927451-84a0-45f0-9ad5-337ba3c8ab14 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:50.579266 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 855/1723] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:41:50 2022] PUT /identity/v3/projects/c61f19562a264e36b2c6f2df0d1c8013/users/ef12528aa8d34059887a81df61e47833/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:41:50.591747 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520df47c-b1ac-449c-88ea-9ab6a3e1ea93 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:50.592283 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520df47c-b1ac-449c-88ea-9ab6a3e1ea93 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:50.592653 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520df47c-b1ac-449c-88ea-9ab6a3e1ea93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:50.601970 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b927451-84a0-45f0-9ad5-337ba3c8ab14 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:50.630079 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:41:50.685283 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-520df47c-b1ac-449c-88ea-9ab6a3e1ea93 None None] MFA Rules not processed for user `ef12528aa8d34059887a81df61e47833`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:50.692010 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-520df47c-b1ac-449c-88ea-9ab6a3e1ea93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:50.760409 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c54d3bb-6532-4c91-a01b-d3c400a41079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:50.764303 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54d3bb-6532-4c91-a01b-d3c400a41079 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:50.765009 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54d3bb-6532-4c91-a01b-d3c400a41079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:50.765301 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54d3bb-6532-4c91-a01b-d3c400a41079 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:50.767630 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c54d3bb-6532-4c91-a01b-d3c400a41079 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:50.770842 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c54d3bb-6532-4c91-a01b-d3c400a41079 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:50.797678 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 869/1724] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:41:50 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:41:50.801651 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c54d3bb-6532-4c91-a01b-d3c400a41079 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:50.826924 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c54d3bb-6532-4c91-a01b-d3c400a41079 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:50.842864 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7242ef9b-f9f9-4c11-898c-238272062063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:50.844084 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242ef9b-f9f9-4c11-898c-238272062063 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:50.845703 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242ef9b-f9f9-4c11-898c-238272062063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:50.846169 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242ef9b-f9f9-4c11-898c-238272062063 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:50.847998 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7242ef9b-f9f9-4c11-898c-238272062063 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:50.853684 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7242ef9b-f9f9-4c11-898c-238272062063 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:50.887582 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 870/1725] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:41:50 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:41:50.888588 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7242ef9b-f9f9-4c11-898c-238272062063 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:50.891476 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 856/1726] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:50 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:41:50.892955 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520df47c-b1ac-449c-88ea-9ab6a3e1ea93 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:50.894703 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-520df47c-b1ac-449c-88ea-9ab6a3e1ea93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:50.902290 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7242ef9b-f9f9-4c11-898c-238272062063 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:50.910354 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b5cda3-9780-4623-a306-ee939524a2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:50.912603 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b5cda3-9780-4623-a306-ee939524a2d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:50.912986 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b5cda3-9780-4623-a306-ee939524a2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:50.913371 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b5cda3-9780-4623-a306-ee939524a2d3 admin admin] PATH_INFO: `/v3/projects/0473cb66299d4f06b36509539ccd6795/users/c9f370738105415187561e9d2ee3d5cc/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:50.980019 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b5cda3-9780-4623-a306-ee939524a2d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0473cb66299d4f06b36509539ccd6795, user_id=c9f370738105415187561e9d2ee3d5cc, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:50.983840 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b5cda3-9780-4623-a306-ee939524a2d3 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:51.010794 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 871/1727] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:41:50 2022] PUT /identity/v3/projects/0473cb66299d4f06b36509539ccd6795/users/c9f370738105415187561e9d2ee3d5cc/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:41:51.022537 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b6c39-2c9d-4d40-ac97-b2fe8fed7943 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:51.023142 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b6c39-2c9d-4d40-ac97-b2fe8fed7943 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:51.023653 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b6c39-2c9d-4d40-ac97-b2fe8fed7943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:51.024445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520df47c-b1ac-449c-88ea-9ab6a3e1ea93 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:51.053352 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:41:51.107731 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-6b6b6c39-2c9d-4d40-ac97-b2fe8fed7943 None None] MFA Rules not processed for user `c9f370738105415187561e9d2ee3d5cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:51.109107 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6b6b6c39-2c9d-4d40-ac97-b2fe8fed7943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:51.137614 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbfbd96-7482-4d3d-975d-460da8a5b11a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:51.138645 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbfbd96-7482-4d3d-975d-460da8a5b11a service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:51.138930 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbfbd96-7482-4d3d-975d-460da8a5b11a service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:51.139427 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbfbd96-7482-4d3d-975d-460da8a5b11a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:51.309548 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 872/1728] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:51.325109 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b6c39-2c9d-4d40-ac97-b2fe8fed7943 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:51.326956 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6b6b6c39-2c9d-4d40-ac97-b2fe8fed7943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:51.327421 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbfbd96-7482-4d3d-975d-460da8a5b11a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:51.329370 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbfbd96-7482-4d3d-975d-460da8a5b11a service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:51.344496 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b6c39-2c9d-4d40-ac97-b2fe8fed7943 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:51.357133 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 857/1729] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:41:50 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:41:51.370091 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d911ad-7771-44f3-94b0-48e544a12c46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:51.370959 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d911ad-7771-44f3-94b0-48e544a12c46 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:51.374249 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d911ad-7771-44f3-94b0-48e544a12c46 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:51.374644 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d911ad-7771-44f3-94b0-48e544a12c46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:51.426523 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d911ad-7771-44f3-94b0-48e544a12c46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:51.432044 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d911ad-7771-44f3-94b0-48e544a12c46 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:51.483268 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 873/1730] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:41:51 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:41:51.501958 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbfbd96-7482-4d3d-975d-460da8a5b11a service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:51.742082 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbfbd96-7482-4d3d-975d-460da8a5b11a service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:51.755663 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea1119f-bd7d-4c46-b219-6025d2aef4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:51.756642 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1119f-bd7d-4c46-b219-6025d2aef4b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:51.756855 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1119f-bd7d-4c46-b219-6025d2aef4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:51.757242 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1119f-bd7d-4c46-b219-6025d2aef4b4 admin admin] PATH_INFO: `/v3/users/c9f370738105415187561e9d2ee3d5cc` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:51.759815 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea1119f-bd7d-4c46-b219-6025d2aef4b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9f370738105415187561e9d2ee3d5cc)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:51.766422 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea1119f-bd7d-4c46-b219-6025d2aef4b4 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:51.956707 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-aea1119f-bd7d-4c46-b219-6025d2aef4b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9f370738105415187561e9d2ee3d5cc'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:51.988983 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-aea1119f-bd7d-4c46-b219-6025d2aef4b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9f370738105415187561e9d2ee3d5cc'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:51.989637 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-aea1119f-bd7d-4c46-b219-6025d2aef4b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9f370738105415187561e9d2ee3d5cc'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:51.994680 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:41:52.005717 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-aea1119f-bd7d-4c46-b219-6025d2aef4b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9f370738105415187561e9d2ee3d5cc'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:52.006549 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-aea1119f-bd7d-4c46-b219-6025d2aef4b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9f370738105415187561e9d2ee3d5cc'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:52.052846 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 858/1731] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:41:51 2022] DELETE /identity/v3/users/c9f370738105415187561e9d2ee3d5cc => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:41:52.059467 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a95151-5a58-430b-9707-466afc6585ee None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:52.060001 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a95151-5a58-430b-9707-466afc6585ee None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:52.060171 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a95151-5a58-430b-9707-466afc6585ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:52.117149 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-82a95151-5a58-430b-9707-466afc6585ee None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:52.140675 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-82a95151-5a58-430b-9707-466afc6585ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:52.410143 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:41:52.477629 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 874/1732] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:52.498381 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea1119f-bd7d-4c46-b219-6025d2aef4b4 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:52.499927 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-aea1119f-bd7d-4c46-b219-6025d2aef4b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:52.610120 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a95151-5a58-430b-9707-466afc6585ee None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:52.611560 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-82a95151-5a58-430b-9707-466afc6585ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:52.636106 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a95151-5a58-430b-9707-466afc6585ee None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:52.665725 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c2271b-ef80-4a36-9490-57076940b5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:52.666426 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c2271b-ef80-4a36-9490-57076940b5da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:52.670684 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c2271b-ef80-4a36-9490-57076940b5da admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:52.670971 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c2271b-ef80-4a36-9490-57076940b5da admin admin] PATH_INFO: `/v3/users/e42b0ef560e14681876d635648cc01fc` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:52.674572 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c2271b-ef80-4a36-9490-57076940b5da admin admin] RBAC: Authorizing `identity:delete_user(user_id=e42b0ef560e14681876d635648cc01fc)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:52.687116 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c2271b-ef80-4a36-9490-57076940b5da admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:52.748173 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea1119f-bd7d-4c46-b219-6025d2aef4b4 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:52.752287 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef45940-223f-457a-a622-b4cbd6b4d7bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:52.753134 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef45940-223f-457a-a622-b4cbd6b4d7bc service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:52.753519 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef45940-223f-457a-a622-b4cbd6b4d7bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:52.754043 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef45940-223f-457a-a622-b4cbd6b4d7bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:52.775220 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef45940-223f-457a-a622-b4cbd6b4d7bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:52.776661 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef45940-223f-457a-a622-b4cbd6b4d7bc service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:52.805615 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 859/1733] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:41:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:41:52.834518 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-80c2271b-ef80-4a36-9490-57076940b5da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e42b0ef560e14681876d635648cc01fc'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:52.841801 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-80c2271b-ef80-4a36-9490-57076940b5da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e42b0ef560e14681876d635648cc01fc'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:52.900024 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-80c2271b-ef80-4a36-9490-57076940b5da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e42b0ef560e14681876d635648cc01fc'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:52.914000 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-80c2271b-ef80-4a36-9490-57076940b5da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e42b0ef560e14681876d635648cc01fc'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:52.933696 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-80c2271b-ef80-4a36-9490-57076940b5da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e42b0ef560e14681876d635648cc01fc'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:52.945003 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 875/1734] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:41:52 2022] DELETE /identity/v3/users/e42b0ef560e14681876d635648cc01fc => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:41:52.947098 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e01d70-dd27-4d9c-9aa3-e6c7f3cf9ae0 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:52.947405 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e01d70-dd27-4d9c-9aa3-e6c7f3cf9ae0 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:52.947656 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e01d70-dd27-4d9c-9aa3-e6c7f3cf9ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:53.028891 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-18e01d70-dd27-4d9c-9aa3-e6c7f3cf9ae0 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:53.032574 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-18e01d70-dd27-4d9c-9aa3-e6c7f3cf9ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:53.134757 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921a3cc-58ef-441d-aabe-242c8ab71bf6 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:53.135503 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921a3cc-58ef-441d-aabe-242c8ab71bf6 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:53.135503 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921a3cc-58ef-441d-aabe-242c8ab71bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:53.190691 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 860/1735] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:53.267760 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-8921a3cc-58ef-441d-aabe-242c8ab71bf6 None None] MFA Rules not processed for user `c81e5fb007614cf2992f5e02e85d6307`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:53.273173 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8921a3cc-58ef-441d-aabe-242c8ab71bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:53.405241 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e01d70-dd27-4d9c-9aa3-e6c7f3cf9ae0 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:53.407258 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-18e01d70-dd27-4d9c-9aa3-e6c7f3cf9ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:53.467385 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 876/1736] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Sep 28 15:41:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:41:53.650378 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e01d70-dd27-4d9c-9aa3-e6c7f3cf9ae0 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:53.655876 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece3860-5118-4a00-9377-4d8d075de976 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:53.657703 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece3860-5118-4a00-9377-4d8d075de976 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:53.658204 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece3860-5118-4a00-9377-4d8d075de976 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:53.658987 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece3860-5118-4a00-9377-4d8d075de976 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:53.661916 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8921a3cc-58ef-441d-aabe-242c8ab71bf6 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:53.663944 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8921a3cc-58ef-441d-aabe-242c8ab71bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:53.675625 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ece3860-5118-4a00-9377-4d8d075de976 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:53.677943 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ece3860-5118-4a00-9377-4d8d075de976 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:53.713229 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 861/1737] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:41:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:41:53.754430 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:41:53.868646 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8921a3cc-58ef-441d-aabe-242c8ab71bf6 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:53.892851 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf9ae98-dd6c-404b-adb2-6162f0603adf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '328051fe7607464f8f5dc0dcc01a817a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:53.895223 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9ae98-dd6c-404b-adb2-6162f0603adf service swift] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:53.895532 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9ae98-dd6c-404b-adb2-6162f0603adf service swift] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:53.895938 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9ae98-dd6c-404b-adb2-6162f0603adf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:53.927712 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf9ae98-dd6c-404b-adb2-6162f0603adf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:53.940140 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf9ae98-dd6c-404b-adb2-6162f0603adf service swift] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:53.982744 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 877/1738] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 15:41:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 28 15:41:56.011059 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece3860-5118-4a00-9377-4d8d075de976 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:56.013919 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7ece3860-5118-4a00-9377-4d8d075de976 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:56.047044 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece3860-5118-4a00-9377-4d8d075de976 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:56.091886 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a9e548-0378-492e-97cc-b133488e52f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:56.092549 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a9e548-0378-492e-97cc-b133488e52f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:56.092832 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a9e548-0378-492e-97cc-b133488e52f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:56.093127 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a9e548-0378-492e-97cc-b133488e52f4 admin admin] PATH_INFO: `/v3/projects/0473cb66299d4f06b36509539ccd6795` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:56.108448 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a9e548-0378-492e-97cc-b133488e52f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0473cb66299d4f06b36509539ccd6795)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:56.110676 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a9e548-0378-492e-97cc-b133488e52f4 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:56.223299 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:41:56.360318 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c2a9e548-0378-492e-97cc-b133488e52f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0473cb66299d4f06b36509539ccd6795', 'request_id': 'req-c2a9e548-0378-492e-97cc-b133488e52f4'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:56.403420 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c2a9e548-0378-492e-97cc-b133488e52f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0473cb66299d4f06b36509539ccd6795', 'request_id': 'req-c2a9e548-0378-492e-97cc-b133488e52f4'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:56.436695 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c2a9e548-0378-492e-97cc-b133488e52f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0473cb66299d4f06b36509539ccd6795', 'request_id': 'req-c2a9e548-0378-492e-97cc-b133488e52f4'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:56.483926 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c2a9e548-0378-492e-97cc-b133488e52f4 admin admin] The token cache is being invalidate because project 0473cb66299d4f06b36509539ccd6795 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:41:56.484670 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c2a9e548-0378-492e-97cc-b133488e52f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:56.490277 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c2a9e548-0378-492e-97cc-b133488e52f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:56.496196 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 862/1739] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:41:56 2022] DELETE /identity/v3/projects/0473cb66299d4f06b36509539ccd6795 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:41:56.502182 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c5891-ce1f-470f-adb5-1aa96fb366e2 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:56.503270 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c5891-ce1f-470f-adb5-1aa96fb366e2 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:56.504065 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c5891-ce1f-470f-adb5-1aa96fb366e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:56.575505 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-ef4c5891-ce1f-470f-adb5-1aa96fb366e2 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:56.578436 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ef4c5891-ce1f-470f-adb5-1aa96fb366e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:56.867459 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 878/1740] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:56.876440 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4162d3-8d58-4a47-978c-a58fdce9aba8 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:56.877026 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4162d3-8d58-4a47-978c-a58fdce9aba8 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:56.877518 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4162d3-8d58-4a47-978c-a58fdce9aba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:56.943608 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-3d4162d3-8d58-4a47-978c-a58fdce9aba8 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:56.946985 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3d4162d3-8d58-4a47-978c-a58fdce9aba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:56.978056 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 863/1741] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:56.980774 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4c5891-ce1f-470f-adb5-1aa96fb366e2 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:56.996097 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4c5891-ce1f-470f-adb5-1aa96fb366e2 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:57.014255 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d703e-0a9a-4bfe-b22b-00d1999f9a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:57.015093 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d703e-0a9a-4bfe-b22b-00d1999f9a38 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:57.015554 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d703e-0a9a-4bfe-b22b-00d1999f9a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:57.015927 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d703e-0a9a-4bfe-b22b-00d1999f9a38 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:57.016656 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6d703e-0a9a-4bfe-b22b-00d1999f9a38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:41:57.017072 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6d703e-0a9a-4bfe-b22b-00d1999f9a38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:57.018864 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6d703e-0a9a-4bfe-b22b-00d1999f9a38 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:57.051191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 879/1742] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:41:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:41:57.053936 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4162d3-8d58-4a47-978c-a58fdce9aba8 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:57.071484 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4162d3-8d58-4a47-978c-a58fdce9aba8 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:57.087070 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b821d63-cf1d-40f6-8fcd-98956bbfbad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:57.087848 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b821d63-cf1d-40f6-8fcd-98956bbfbad4 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:57.088230 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b821d63-cf1d-40f6-8fcd-98956bbfbad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:57.088619 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b821d63-cf1d-40f6-8fcd-98956bbfbad4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:57.089391 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b821d63-cf1d-40f6-8fcd-98956bbfbad4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:57.093517 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b821d63-cf1d-40f6-8fcd-98956bbfbad4 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:57.171548 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 864/1743] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:41:57 2022] POST /identity/v3/projects => generated 371 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:41:57.175619 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d703e-0a9a-4bfe-b22b-00d1999f9a38 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:57.187628 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d703e-0a9a-4bfe-b22b-00d1999f9a38 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:57.333034 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5ae711-d64f-4731-a529-847a4e61d58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:57.334016 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ae711-d64f-4731-a529-847a4e61d58e admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:57.334173 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ae711-d64f-4731-a529-847a4e61d58e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:57.334512 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5ae711-d64f-4731-a529-847a4e61d58e admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:57.335222 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5ae711-d64f-4731-a529-847a4e61d58e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:57.337289 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5ae711-d64f-4731-a529-847a4e61d58e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:57.412156 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 880/1744] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:41:57 2022] POST /identity/v3/users => generated 286 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:41:57.416454 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b821d63-cf1d-40f6-8fcd-98956bbfbad4 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:57.435142 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b821d63-cf1d-40f6-8fcd-98956bbfbad4 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:57.458198 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5d665c-63f2-401d-909e-96039601cb43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:57.458937 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d665c-63f2-401d-909e-96039601cb43 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:57.468048 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d665c-63f2-401d-909e-96039601cb43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:57.468423 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d665c-63f2-401d-909e-96039601cb43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:57.469372 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5d665c-63f2-401d-909e-96039601cb43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:57.471254 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5d665c-63f2-401d-909e-96039601cb43 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:57.487513 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 865/1745] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:41:57.491558 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5ae711-d64f-4731-a529-847a4e61d58e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:57.522172 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5ae711-d64f-4731-a529-847a4e61d58e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:57.537087 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c1874f-e3aa-4ca0-a299-df3b0cd25a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:57.538018 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c1874f-e3aa-4ca0-a299-df3b0cd25a0e admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:57.538461 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c1874f-e3aa-4ca0-a299-df3b0cd25a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:57.538881 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c1874f-e3aa-4ca0-a299-df3b0cd25a0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:57.539668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c1874f-e3aa-4ca0-a299-df3b0cd25a0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:57.541425 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c1874f-e3aa-4ca0-a299-df3b0cd25a0e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:57.573738 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5d665c-63f2-401d-909e-96039601cb43 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:57.575287 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 881/1746] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:41:57.591118 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5d665c-63f2-401d-909e-96039601cb43 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:57.602592 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a15e630-0169-4067-b492-48a2cf6378f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:57.603394 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a15e630-0169-4067-b492-48a2cf6378f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:57.603768 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a15e630-0169-4067-b492-48a2cf6378f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:57.604136 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a15e630-0169-4067-b492-48a2cf6378f5 admin admin] PATH_INFO: `/v3/projects/f9b4928b5e2d4f6286e8e56d35c813a9/users/5558f9c2bb37403b9ff03f2be1aa3113/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:57.643594 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a15e630-0169-4067-b492-48a2cf6378f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9b4928b5e2d4f6286e8e56d35c813a9, user_id=5558f9c2bb37403b9ff03f2be1aa3113, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:57.646685 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a15e630-0169-4067-b492-48a2cf6378f5 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:57.669949 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 866/1747] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:41:57 2022] PUT /identity/v3/projects/f9b4928b5e2d4f6286e8e56d35c813a9/users/5558f9c2bb37403b9ff03f2be1aa3113/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:41:57.672076 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564ad6e-4667-46ca-9428-5660beaa9ecc None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:57.673083 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564ad6e-4667-46ca-9428-5660beaa9ecc None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:57.673516 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564ad6e-4667-46ca-9428-5660beaa9ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:57.729931 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-5564ad6e-4667-46ca-9428-5660beaa9ecc None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:57.733578 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5564ad6e-4667-46ca-9428-5660beaa9ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:57.748966 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:41:57.803850 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a15e630-0169-4067-b492-48a2cf6378f5 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:57.806169 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-2a15e630-0169-4067-b492-48a2cf6378f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:57.891633 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 882/1748] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:57.899500 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564ad6e-4667-46ca-9428-5660beaa9ecc None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:57.901637 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5564ad6e-4667-46ca-9428-5660beaa9ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:57.973284 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a15e630-0169-4067-b492-48a2cf6378f5 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:57.978276 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a509fa2-937c-4524-bfa3-0abe6faded24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:57.979088 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a509fa2-937c-4524-bfa3-0abe6faded24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:57.979524 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a509fa2-937c-4524-bfa3-0abe6faded24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:57.980864 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a509fa2-937c-4524-bfa3-0abe6faded24 admin admin] PATH_INFO: `/v3/projects/3493d310d37d4ff6a2525b58f9fe9609` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:57.982246 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a509fa2-937c-4524-bfa3-0abe6faded24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3493d310d37d4ff6a2525b58f9fe9609)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:57.986786 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a509fa2-937c-4524-bfa3-0abe6faded24 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:58.030114 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564ad6e-4667-46ca-9428-5660beaa9ecc None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:58.055954 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23320260-80a2-49ee-af2b-246be3247635 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:58.056642 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23320260-80a2-49ee-af2b-246be3247635 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:58.057038 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23320260-80a2-49ee-af2b-246be3247635 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:58.069229 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23320260-80a2-49ee-af2b-246be3247635 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:58.083398 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23320260-80a2-49ee-af2b-246be3247635 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:58.086988 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23320260-80a2-49ee-af2b-246be3247635 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:58.116202 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:41:58.122117 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 883/1749] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:41:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:41:58.123405 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4a509fa2-937c-4524-bfa3-0abe6faded24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3493d310d37d4ff6a2525b58f9fe9609', 'request_id': 'req-4a509fa2-937c-4524-bfa3-0abe6faded24'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:58.139265 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4a509fa2-937c-4524-bfa3-0abe6faded24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3493d310d37d4ff6a2525b58f9fe9609', 'request_id': 'req-4a509fa2-937c-4524-bfa3-0abe6faded24'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:58.150036 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4a509fa2-937c-4524-bfa3-0abe6faded24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3493d310d37d4ff6a2525b58f9fe9609', 'request_id': 'req-4a509fa2-937c-4524-bfa3-0abe6faded24'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:58.164262 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4a509fa2-937c-4524-bfa3-0abe6faded24 admin admin] The token cache is being invalidate because project 3493d310d37d4ff6a2525b58f9fe9609 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:41:58.164578 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4a509fa2-937c-4524-bfa3-0abe6faded24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:58.165569 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4a509fa2-937c-4524-bfa3-0abe6faded24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:58.167306 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 867/1750] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:41:57 2022] DELETE /identity/v3/projects/3493d310d37d4ff6a2525b58f9fe9609 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:41:58.171462 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23320260-80a2-49ee-af2b-246be3247635 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:58.185422 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23320260-80a2-49ee-af2b-246be3247635 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:58.289247 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ed554-d609-4c45-bfea-dd2b2bb9166e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:58.290490 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ed554-d609-4c45-bfea-dd2b2bb9166e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:58.290864 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ed554-d609-4c45-bfea-dd2b2bb9166e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:58.291485 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ed554-d609-4c45-bfea-dd2b2bb9166e admin admin] PATH_INFO: `/v3/users/ef12528aa8d34059887a81df61e47833` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:58.294505 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64ed554-d609-4c45-bfea-dd2b2bb9166e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef12528aa8d34059887a81df61e47833)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:58.298291 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64ed554-d609-4c45-bfea-dd2b2bb9166e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:58.387439 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-f64ed554-d609-4c45-bfea-dd2b2bb9166e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef12528aa8d34059887a81df61e47833'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:58.388634 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-f64ed554-d609-4c45-bfea-dd2b2bb9166e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef12528aa8d34059887a81df61e47833'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:58.405533 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:41:58.418164 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-f64ed554-d609-4c45-bfea-dd2b2bb9166e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef12528aa8d34059887a81df61e47833'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:58.433906 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-f64ed554-d609-4c45-bfea-dd2b2bb9166e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef12528aa8d34059887a81df61e47833'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:58.447675 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-f64ed554-d609-4c45-bfea-dd2b2bb9166e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef12528aa8d34059887a81df61e47833'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:58.474356 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 884/1751] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:41:58 2022] DELETE /identity/v3/users/ef12528aa8d34059887a81df61e47833 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:41:59.055296 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a509fa2-937c-4524-bfa3-0abe6faded24 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:59.058726 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4713cbf8-5f3b-45ea-866d-683112ec7379 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:59.059192 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4713cbf8-5f3b-45ea-866d-683112ec7379 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:59.060940 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4713cbf8-5f3b-45ea-866d-683112ec7379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:59.063766 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4a509fa2-937c-4524-bfa3-0abe6faded24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:59.095173 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a509fa2-937c-4524-bfa3-0abe6faded24 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:59.137331 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-4713cbf8-5f3b-45ea-866d-683112ec7379 None None] MFA Rules not processed for user `5558f9c2bb37403b9ff03f2be1aa3113`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:59.138642 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4713cbf8-5f3b-45ea-866d-683112ec7379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:59.209615 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61780179-b5a9-4396-9c52-19886e14d1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:59.224964 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61780179-b5a9-4396-9c52-19886e14d1e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:59.225588 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61780179-b5a9-4396-9c52-19886e14d1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:59.226474 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61780179-b5a9-4396-9c52-19886e14d1e4 admin admin] PATH_INFO: `/v3/projects/c61f19562a264e36b2c6f2df0d1c8013` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:59.228493 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61780179-b5a9-4396-9c52-19886e14d1e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c61f19562a264e36b2c6f2df0d1c8013)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:59.230827 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61780179-b5a9-4396-9c52-19886e14d1e4 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:59.253953 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 885/1752] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:59.264064 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4713cbf8-5f3b-45ea-866d-683112ec7379 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:59.266019 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4713cbf8-5f3b-45ea-866d-683112ec7379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:59.346665 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-61780179-b5a9-4396-9c52-19886e14d1e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c61f19562a264e36b2c6f2df0d1c8013', 'request_id': 'req-61780179-b5a9-4396-9c52-19886e14d1e4'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:59.363666 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-61780179-b5a9-4396-9c52-19886e14d1e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c61f19562a264e36b2c6f2df0d1c8013', 'request_id': 'req-61780179-b5a9-4396-9c52-19886e14d1e4'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:59.376013 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-61780179-b5a9-4396-9c52-19886e14d1e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c61f19562a264e36b2c6f2df0d1c8013', 'request_id': 'req-61780179-b5a9-4396-9c52-19886e14d1e4'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:59.398185 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-61780179-b5a9-4396-9c52-19886e14d1e4 admin admin] The token cache is being invalidate because project c61f19562a264e36b2c6f2df0d1c8013 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:41:59.398568 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-61780179-b5a9-4396-9c52-19886e14d1e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:59.399468 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-61780179-b5a9-4396-9c52-19886e14d1e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:41:59.407250 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 868/1753] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:41:59 2022] DELETE /identity/v3/projects/c61f19562a264e36b2c6f2df0d1c8013 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:41:59.409027 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f24d6a9-0a81-4804-9e64-c23886f152b5 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:59.409509 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f24d6a9-0a81-4804-9e64-c23886f152b5 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:59.409757 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4713cbf8-5f3b-45ea-866d-683112ec7379 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:59.410273 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f24d6a9-0a81-4804-9e64-c23886f152b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:59.432897 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-8f24d6a9-0a81-4804-9e64-c23886f152b5 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:59.434461 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-8f24d6a9-0a81-4804-9e64-c23886f152b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:59.508824 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8b538a-6a92-40bf-aa5b-a72acc04ad7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:59.509844 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b538a-6a92-40bf-aa5b-a72acc04ad7d service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:59.511717 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b538a-6a92-40bf-aa5b-a72acc04ad7d service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:59.511717 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b538a-6a92-40bf-aa5b-a72acc04ad7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:59.513755 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ad8b538a-6a92-40bf-aa5b-a72acc04ad7d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:59.549574 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 869/1754] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:59.557668 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf1f762-8c2e-4735-a045-67cb96ee10a9 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:59.558221 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf1f762-8c2e-4735-a045-67cb96ee10a9 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:59.558859 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf1f762-8c2e-4735-a045-67cb96ee10a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:59.579930 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:41:59.581205 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:41:59.594870 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-ddf1f762-8c2e-4735-a045-67cb96ee10a9 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:41:59.596639 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ddf1f762-8c2e-4735-a045-67cb96ee10a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:41:59.607767 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 870/1755] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:41:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:41:59.611867 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf1f762-8c2e-4735-a045-67cb96ee10a9 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:59.622730 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8b538a-6a92-40bf-aa5b-a72acc04ad7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:59.624397 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8b538a-6a92-40bf-aa5b-a72acc04ad7d service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:59.636963 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf1f762-8c2e-4735-a045-67cb96ee10a9 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:59.642917 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3373e278-5185-43e6-9c6a-24b04d7e106c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:59.643588 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373e278-5185-43e6-9c6a-24b04d7e106c admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:59.643876 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373e278-5185-43e6-9c6a-24b04d7e106c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:59.644146 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373e278-5185-43e6-9c6a-24b04d7e106c admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:59.644944 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3373e278-5185-43e6-9c6a-24b04d7e106c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:41:59.645245 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3373e278-5185-43e6-9c6a-24b04d7e106c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:59.646874 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 886/1756] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:41:59 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:41:59.657104 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3373e278-5185-43e6-9c6a-24b04d7e106c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:59.674861 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 871/1757] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:41:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:41:59.679242 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8b538a-6a92-40bf-aa5b-a72acc04ad7d service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:59.702096 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8b538a-6a92-40bf-aa5b-a72acc04ad7d service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:41:59.716530 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28f31cc-66a7-4609-a7a1-51fe8f989ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:41:59.717171 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f31cc-66a7-4609-a7a1-51fe8f989ece admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:41:59.719740 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f31cc-66a7-4609-a7a1-51fe8f989ece admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:41:59.719740 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f31cc-66a7-4609-a7a1-51fe8f989ece admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:41:59.720708 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28f31cc-66a7-4609-a7a1-51fe8f989ece admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:41:59.723320 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28f31cc-66a7-4609-a7a1-51fe8f989ece admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:41:59.820974 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3373e278-5185-43e6-9c6a-24b04d7e106c admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:41:59.822268 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 887/1758] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:41:59 2022] POST /identity/v3/projects => generated 357 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:41:59.836187 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3373e278-5185-43e6-9c6a-24b04d7e106c admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:00.021359 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a638531e-9a14-4ae7-974c-4fe27033cd32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:00.025465 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638531e-9a14-4ae7-974c-4fe27033cd32 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:00.028982 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638531e-9a14-4ae7-974c-4fe27033cd32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:00.029391 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638531e-9a14-4ae7-974c-4fe27033cd32 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:00.030484 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a638531e-9a14-4ae7-974c-4fe27033cd32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:00.033648 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a638531e-9a14-4ae7-974c-4fe27033cd32 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:00.099401 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28f31cc-66a7-4609-a7a1-51fe8f989ece admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:00.108853 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 872/1759] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:41:59 2022] POST /identity/v3/users => generated 279 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:00.113008 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28f31cc-66a7-4609-a7a1-51fe8f989ece admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:00.130673 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce44cb12-eab8-43fd-b0d7-1cdc2635af18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:00.131353 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce44cb12-eab8-43fd-b0d7-1cdc2635af18 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:00.131683 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce44cb12-eab8-43fd-b0d7-1cdc2635af18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:00.132024 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce44cb12-eab8-43fd-b0d7-1cdc2635af18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:00.132677 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce44cb12-eab8-43fd-b0d7-1cdc2635af18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:00.134511 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce44cb12-eab8-43fd-b0d7-1cdc2635af18 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:00.158725 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 888/1760] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:00 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:00.166094 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a638531e-9a14-4ae7-974c-4fe27033cd32 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:00.176907 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a638531e-9a14-4ae7-974c-4fe27033cd32 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:00.187010 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00753c58-fb8c-4ff6-86c0-c6481d4aa197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:00.187782 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00753c58-fb8c-4ff6-86c0-c6481d4aa197 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:00.188061 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00753c58-fb8c-4ff6-86c0-c6481d4aa197 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:00.188564 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00753c58-fb8c-4ff6-86c0-c6481d4aa197 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:00.189130 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00753c58-fb8c-4ff6-86c0-c6481d4aa197 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:00.191203 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00753c58-fb8c-4ff6-86c0-c6481d4aa197 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:00.210238 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 873/1761] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:00 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:42:00.214936 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce44cb12-eab8-43fd-b0d7-1cdc2635af18 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:00.235610 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce44cb12-eab8-43fd-b0d7-1cdc2635af18 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:00.248726 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ab6aa2-6772-49a4-9793-227b23a4b87f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:00.249389 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab6aa2-6772-49a4-9793-227b23a4b87f admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:00.249553 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab6aa2-6772-49a4-9793-227b23a4b87f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:00.249937 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab6aa2-6772-49a4-9793-227b23a4b87f admin admin] PATH_INFO: `/v3/projects/3db0be6d2ea24d41af8363cacc9122f8/users/984dcaeea04744dfb8bb7e11017cc1c0/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:00.288754 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ab6aa2-6772-49a4-9793-227b23a4b87f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3db0be6d2ea24d41af8363cacc9122f8, user_id=984dcaeea04744dfb8bb7e11017cc1c0, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:00.295729 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ab6aa2-6772-49a4-9793-227b23a4b87f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:00.322249 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 889/1762] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:00 2022] PUT /identity/v3/projects/3db0be6d2ea24d41af8363cacc9122f8/users/984dcaeea04744dfb8bb7e11017cc1c0/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:00.323603 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a1a75-d4e8-402f-ac58-ba85136287be None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:00.323916 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a1a75-d4e8-402f-ac58-ba85136287be None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:00.324192 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a1a75-d4e8-402f-ac58-ba85136287be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:00.381526 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-237a1a75-d4e8-402f-ac58-ba85136287be None None] MFA Rules not processed for user `984dcaeea04744dfb8bb7e11017cc1c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:00.382868 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-237a1a75-d4e8-402f-ac58-ba85136287be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:00.566157 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 874/1763] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:42:00 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:42:00.571075 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ab6aa2-6772-49a4-9793-227b23a4b87f admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:00.708820 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ab6aa2-6772-49a4-9793-227b23a4b87f admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:00.715641 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454237d6-c5d6-444c-9d79-99a9ad51110e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:00.716596 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454237d6-c5d6-444c-9d79-99a9ad51110e admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:00.717500 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454237d6-c5d6-444c-9d79-99a9ad51110e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:00.717787 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454237d6-c5d6-444c-9d79-99a9ad51110e admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:00.719050 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454237d6-c5d6-444c-9d79-99a9ad51110e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:00.721771 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454237d6-c5d6-444c-9d79-99a9ad51110e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:00.762685 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237a1a75-d4e8-402f-ac58-ba85136287be None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:00.770632 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 890/1764] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:42:00 2022] POST /identity/v3/projects => generated 357 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:42:00.782391 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237a1a75-d4e8-402f-ac58-ba85136287be None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:00.861611 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:00.873561 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92e7f8f-b0ee-41eb-afd4-fecd94260667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:00.874216 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e7f8f-b0ee-41eb-afd4-fecd94260667 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:00.874487 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e7f8f-b0ee-41eb-afd4-fecd94260667 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:00.874753 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e7f8f-b0ee-41eb-afd4-fecd94260667 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:00.875653 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92e7f8f-b0ee-41eb-afd4-fecd94260667 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:00.877567 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92e7f8f-b0ee-41eb-afd4-fecd94260667 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:00.924549 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454237d6-c5d6-444c-9d79-99a9ad51110e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:00.926154 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 875/1765] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:42:00 2022] POST /identity/v3/users => generated 285 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:00.938892 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454237d6-c5d6-444c-9d79-99a9ad51110e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:00.946355 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab86be88-9ec9-4204-b20a-36c9999a00ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:00.947138 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86be88-9ec9-4204-b20a-36c9999a00ba admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:00.947516 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86be88-9ec9-4204-b20a-36c9999a00ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:00.947990 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86be88-9ec9-4204-b20a-36c9999a00ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:00.948751 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab86be88-9ec9-4204-b20a-36c9999a00ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:00.950862 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab86be88-9ec9-4204-b20a-36c9999a00ba admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:00.969380 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 891/1766] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:00 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:00.973266 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92e7f8f-b0ee-41eb-afd4-fecd94260667 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:00.983639 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92e7f8f-b0ee-41eb-afd4-fecd94260667 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:00.988182 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e0432-a19d-4dc6-8681-4002a7c9e6be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:00.988836 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e0432-a19d-4dc6-8681-4002a7c9e6be admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:00.989113 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e0432-a19d-4dc6-8681-4002a7c9e6be admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:00.989489 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e0432-a19d-4dc6-8681-4002a7c9e6be admin admin] PATH_INFO: `/v3/projects/f871eb2d28c340b186085cb0032947fe/users/47e562316d994f1987e5bef47eec3620/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:01.021532 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657e0432-a19d-4dc6-8681-4002a7c9e6be admin admin] RBAC: Authorizing `identity:create_grant(project_id=f871eb2d28c340b186085cb0032947fe, user_id=47e562316d994f1987e5bef47eec3620, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:01.025140 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657e0432-a19d-4dc6-8681-4002a7c9e6be admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:01.051628 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 876/1767] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:00 2022] PUT /identity/v3/projects/f871eb2d28c340b186085cb0032947fe/users/47e562316d994f1987e5bef47eec3620/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:01.055665 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab86be88-9ec9-4204-b20a-36c9999a00ba admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:01.070874 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab86be88-9ec9-4204-b20a-36c9999a00ba admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:01.168784 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:01.213439 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3a8b37-21a9-48a2-9140-5e0820ab749d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:01.214814 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a8b37-21a9-48a2-9140-5e0820ab749d admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:01.215246 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a8b37-21a9-48a2-9140-5e0820ab749d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:01.215667 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a8b37-21a9-48a2-9140-5e0820ab749d admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:01.216465 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3a8b37-21a9-48a2-9140-5e0820ab749d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:01.219695 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3a8b37-21a9-48a2-9140-5e0820ab749d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:01.235305 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 892/1768] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:01 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:01.240069 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e0432-a19d-4dc6-8681-4002a7c9e6be admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:01.265408 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e0432-a19d-4dc6-8681-4002a7c9e6be admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:01.283554 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7ae2f1-8845-4dd0-bea5-8459eba1eac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:01.284244 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7ae2f1-8845-4dd0-bea5-8459eba1eac6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:01.284537 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7ae2f1-8845-4dd0-bea5-8459eba1eac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:01.284810 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7ae2f1-8845-4dd0-bea5-8459eba1eac6 admin admin] PATH_INFO: `/v3/projects/f871eb2d28c340b186085cb0032947fe/users/47e562316d994f1987e5bef47eec3620/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:01.286870 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7ae2f1-8845-4dd0-bea5-8459eba1eac6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f871eb2d28c340b186085cb0032947fe, user_id=47e562316d994f1987e5bef47eec3620, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:01.290247 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7ae2f1-8845-4dd0-bea5-8459eba1eac6 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:01.305229 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 877/1769] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:01 2022] PUT /identity/v3/projects/f871eb2d28c340b186085cb0032947fe/users/47e562316d994f1987e5bef47eec3620/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:01.318715 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0433ca-16a3-4423-ae9e-8c2b7f8076b4 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:01.319193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0433ca-16a3-4423-ae9e-8c2b7f8076b4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:01.319560 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0433ca-16a3-4423-ae9e-8c2b7f8076b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:01.356667 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-3a0433ca-16a3-4423-ae9e-8c2b7f8076b4 None None] MFA Rules not processed for user `47e562316d994f1987e5bef47eec3620`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:01.357912 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3a0433ca-16a3-4423-ae9e-8c2b7f8076b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:01.459198 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 893/1770] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:42:01 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:01.466608 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7ae2f1-8845-4dd0-bea5-8459eba1eac6 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:01.468461 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-fd7ae2f1-8845-4dd0-bea5-8459eba1eac6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:01.544695 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7ae2f1-8845-4dd0-bea5-8459eba1eac6 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:01.554598 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c26ce0-e296-425c-ad66-b267294f957c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:01.555234 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c26ce0-e296-425c-ad66-b267294f957c service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:01.555506 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c26ce0-e296-425c-ad66-b267294f957c service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:01.555782 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c26ce0-e296-425c-ad66-b267294f957c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:01.582297 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c26ce0-e296-425c-ad66-b267294f957c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:01.583745 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c26ce0-e296-425c-ad66-b267294f957c service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:01.595181 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 878/1771] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:01 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:42:01.689191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0433ca-16a3-4423-ae9e-8c2b7f8076b4 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:01.712597 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0433ca-16a3-4423-ae9e-8c2b7f8076b4 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:01.716521 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f88e2fd-036c-494f-ae3f-a320062d8b05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:01.717093 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f88e2fd-036c-494f-ae3f-a320062d8b05 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:01.717365 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f88e2fd-036c-494f-ae3f-a320062d8b05 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:01.717636 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f88e2fd-036c-494f-ae3f-a320062d8b05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:01.817645 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f88e2fd-036c-494f-ae3f-a320062d8b05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:01.819229 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f88e2fd-036c-494f-ae3f-a320062d8b05 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:01.828467 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 894/1772] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:01 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:42:03.487096 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc13cc7-47e4-40ba-ab6a-62fe94012256 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:03.487709 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc13cc7-47e4-40ba-ab6a-62fe94012256 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:03.487794 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc13cc7-47e4-40ba-ab6a-62fe94012256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:03.509734 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-0dc13cc7-47e4-40ba-ab6a-62fe94012256 None None] MFA Rules not processed for user `5558f9c2bb37403b9ff03f2be1aa3113`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:03.511716 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-0dc13cc7-47e4-40ba-ab6a-62fe94012256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:03.602351 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 879/1773] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:42:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:06.471341 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7fee37-e228-40d7-bc61-1accde531af6 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:06.471953 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7fee37-e228-40d7-bc61-1accde531af6 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:06.471953 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7fee37-e228-40d7-bc61-1accde531af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:06.494851 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-0e7fee37-e228-40d7-bc61-1accde531af6 None None] MFA Rules not processed for user `b13b0533444a449ead00a389c5f6bd1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:06.496771 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0e7fee37-e228-40d7-bc61-1accde531af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:06.571482 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 895/1774] 10.222.0.50 () {58 vars in 1015 bytes} [Wed Sep 28 15:42:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:06.626238 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc13cc7-47e4-40ba-ab6a-62fe94012256 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:06.626238 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-0dc13cc7-47e4-40ba-ab6a-62fe94012256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:06.668562 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc13cc7-47e4-40ba-ab6a-62fe94012256 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:06.682174 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fae0c0-80dd-423e-a5eb-14ee1eb4107c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:06.688939 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fae0c0-80dd-423e-a5eb-14ee1eb4107c service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:06.689436 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fae0c0-80dd-423e-a5eb-14ee1eb4107c service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:06.690066 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fae0c0-80dd-423e-a5eb-14ee1eb4107c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:06.725628 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fae0c0-80dd-423e-a5eb-14ee1eb4107c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:06.725628 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fae0c0-80dd-423e-a5eb-14ee1eb4107c service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:06.756108 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 880/1775] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:42:09.114783 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367190ea-8ef5-43c9-a141-20e2fe794d55 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:09.115300 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367190ea-8ef5-43c9-a141-20e2fe794d55 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:09.115680 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367190ea-8ef5-43c9-a141-20e2fe794d55 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:09.117796 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 896/1776] 10.222.0.52 () {54 vars in 921 bytes} [Wed Sep 28 15:42:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:42:09.156427 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b2c5f-6267-437b-bb68-d71d6f67efa6 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:09.156924 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b2c5f-6267-437b-bb68-d71d6f67efa6 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:09.157087 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b2c5f-6267-437b-bb68-d71d6f67efa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:09.177809 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-0f8b2c5f-6267-437b-bb68-d71d6f67efa6 None None] MFA Rules not processed for user `8daf14b64bf04b2ba74d2fe1fd41ff71`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:09.179493 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-0f8b2c5f-6267-437b-bb68-d71d6f67efa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:09.220745 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:09.261618 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 881/1777] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Sep 28 15:42:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:09.867101 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367190ea-8ef5-43c9-a141-20e2fe794d55 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:09.868921 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-367190ea-8ef5-43c9-a141-20e2fe794d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:09.909215 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367190ea-8ef5-43c9-a141-20e2fe794d55 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:09.922933 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baea730-7aa2-4fca-986b-bf5fdad0026d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf14b64bf04b2ba74d2fe1fd41ff71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:09.925054 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baea730-7aa2-4fca-986b-bf5fdad0026d service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:09.925337 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baea730-7aa2-4fca-986b-bf5fdad0026d service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:09.928945 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baea730-7aa2-4fca-986b-bf5fdad0026d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:09.948058 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baea730-7aa2-4fca-986b-bf5fdad0026d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:09.956406 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baea730-7aa2-4fca-986b-bf5fdad0026d service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:09.977592 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 897/1778] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:42:10.839247 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a8b48e-7bb7-4c5a-8ba0-b2bba06a0894 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:10.840255 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a8b48e-7bb7-4c5a-8ba0-b2bba06a0894 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:10.840751 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a8b48e-7bb7-4c5a-8ba0-b2bba06a0894 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:10.842848 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 882/1779] 10.222.0.52 () {54 vars in 921 bytes} [Wed Sep 28 15:42:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:42:10.882783 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ada451-7966-4b37-94d2-437c81d7eb57 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:10.884422 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ada451-7966-4b37-94d2-437c81d7eb57 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:10.887558 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ada451-7966-4b37-94d2-437c81d7eb57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:10.905547 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:10.919881 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-c7ada451-7966-4b37-94d2-437c81d7eb57 None None] MFA Rules not processed for user `8daf14b64bf04b2ba74d2fe1fd41ff71`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:10.921476 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c7ada451-7966-4b37-94d2-437c81d7eb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:10.934540 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 898/1780] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Sep 28 15:42:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:11.528614 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a8b48e-7bb7-4c5a-8ba0-b2bba06a0894 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:11.565068 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a8b48e-7bb7-4c5a-8ba0-b2bba06a0894 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:11.581133 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590161fa-17f9-4a63-b59e-8db61970dd63 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf14b64bf04b2ba74d2fe1fd41ff71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:11.583934 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590161fa-17f9-4a63-b59e-8db61970dd63 service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:11.584417 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590161fa-17f9-4a63-b59e-8db61970dd63 service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:11.584988 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590161fa-17f9-4a63-b59e-8db61970dd63 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:11.600227 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590161fa-17f9-4a63-b59e-8db61970dd63 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:11.601831 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590161fa-17f9-4a63-b59e-8db61970dd63 service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:11.615148 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 883/1781] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:42:16.880050 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ada451-7966-4b37-94d2-437c81d7eb57 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:16.881875 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c7ada451-7966-4b37-94d2-437c81d7eb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:16.978183 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ada451-7966-4b37-94d2-437c81d7eb57 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:16.984380 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77430da6-61cc-469b-84b1-56c8257b22cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:16.985277 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77430da6-61cc-469b-84b1-56c8257b22cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:16.985814 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77430da6-61cc-469b-84b1-56c8257b22cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:16.986482 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77430da6-61cc-469b-84b1-56c8257b22cd admin admin] PATH_INFO: `/v3/users/957373f98691469386b41fe0e17e4bd3` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:16.988053 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77430da6-61cc-469b-84b1-56c8257b22cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=957373f98691469386b41fe0e17e4bd3)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:16.990710 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77430da6-61cc-469b-84b1-56c8257b22cd admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:17.074908 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-77430da6-61cc-469b-84b1-56c8257b22cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '957373f98691469386b41fe0e17e4bd3'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:17.075666 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-77430da6-61cc-469b-84b1-56c8257b22cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '957373f98691469386b41fe0e17e4bd3'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:17.090913 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-77430da6-61cc-469b-84b1-56c8257b22cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '957373f98691469386b41fe0e17e4bd3'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:17.099729 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-77430da6-61cc-469b-84b1-56c8257b22cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '957373f98691469386b41fe0e17e4bd3'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:17.109817 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-77430da6-61cc-469b-84b1-56c8257b22cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '957373f98691469386b41fe0e17e4bd3'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:17.116941 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 899/1782] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:42:16 2022] DELETE /identity/v3/users/957373f98691469386b41fe0e17e4bd3 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:17.122271 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac62a4a-f2e7-4552-b312-126d5d5e09d3 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:17.122731 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac62a4a-f2e7-4552-b312-126d5d5e09d3 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:17.122825 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac62a4a-f2e7-4552-b312-126d5d5e09d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:17.143369 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-8ac62a4a-f2e7-4552-b312-126d5d5e09d3 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:17.145158 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-8ac62a4a-f2e7-4552-b312-126d5d5e09d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:17.212724 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 884/1783] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:42:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:17.243918 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77430da6-61cc-469b-84b1-56c8257b22cd admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:17.245401 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-77430da6-61cc-469b-84b1-56c8257b22cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:17.332543 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77430da6-61cc-469b-84b1-56c8257b22cd admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:17.336188 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7791c46-5462-4e5d-9288-c4e7f65ace38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:17.336812 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7791c46-5462-4e5d-9288-c4e7f65ace38 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:17.337091 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7791c46-5462-4e5d-9288-c4e7f65ace38 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:17.337363 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7791c46-5462-4e5d-9288-c4e7f65ace38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:17.347421 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7791c46-5462-4e5d-9288-c4e7f65ace38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:17.348877 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7791c46-5462-4e5d-9288-c4e7f65ace38 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:17.360062 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 900/1784] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:42:18.067051 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac62a4a-f2e7-4552-b312-126d5d5e09d3 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:18.068948 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-8ac62a4a-f2e7-4552-b312-126d5d5e09d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:18.086941 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac62a4a-f2e7-4552-b312-126d5d5e09d3 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:18.093816 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d43315f-b512-47d1-9713-39b71674f0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:18.095134 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43315f-b512-47d1-9713-39b71674f0d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:18.095579 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43315f-b512-47d1-9713-39b71674f0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:18.095988 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43315f-b512-47d1-9713-39b71674f0d3 admin admin] PATH_INFO: `/v3/projects/972cdebd72ff41c0aa6a092d6d93693e` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:18.097344 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d43315f-b512-47d1-9713-39b71674f0d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=972cdebd72ff41c0aa6a092d6d93693e)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:18.099517 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d43315f-b512-47d1-9713-39b71674f0d3 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:18.155578 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1d43315f-b512-47d1-9713-39b71674f0d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '972cdebd72ff41c0aa6a092d6d93693e', 'request_id': 'req-1d43315f-b512-47d1-9713-39b71674f0d3'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:18.164388 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1d43315f-b512-47d1-9713-39b71674f0d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '972cdebd72ff41c0aa6a092d6d93693e', 'request_id': 'req-1d43315f-b512-47d1-9713-39b71674f0d3'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:18.174923 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1d43315f-b512-47d1-9713-39b71674f0d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '972cdebd72ff41c0aa6a092d6d93693e', 'request_id': 'req-1d43315f-b512-47d1-9713-39b71674f0d3'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:18.190833 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1d43315f-b512-47d1-9713-39b71674f0d3 admin admin] The token cache is being invalidate because project 972cdebd72ff41c0aa6a092d6d93693e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:42:18.191144 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1d43315f-b512-47d1-9713-39b71674f0d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:18.191841 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1d43315f-b512-47d1-9713-39b71674f0d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:18.193369 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 885/1785] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:42:18 2022] DELETE /identity/v3/projects/972cdebd72ff41c0aa6a092d6d93693e => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:18.197306 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7791c46-5462-4e5d-9288-c4e7f65ace38 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:18.209804 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7791c46-5462-4e5d-9288-c4e7f65ace38 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:18.231421 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:18.336824 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2aa5ac-b3bb-4b3d-9a6f-d5065f4356ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:18.337551 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2aa5ac-b3bb-4b3d-9a6f-d5065f4356ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:18.337932 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2aa5ac-b3bb-4b3d-9a6f-d5065f4356ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:18.338550 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2aa5ac-b3bb-4b3d-9a6f-d5065f4356ab admin admin] PATH_INFO: `/v3/users/a7847497073c46afbeaa601617115b06` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:18.339555 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2aa5ac-b3bb-4b3d-9a6f-d5065f4356ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7847497073c46afbeaa601617115b06)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:18.341557 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2aa5ac-b3bb-4b3d-9a6f-d5065f4356ab admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:18.461032 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1f2aa5ac-b3bb-4b3d-9a6f-d5065f4356ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7847497073c46afbeaa601617115b06'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:18.462985 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1f2aa5ac-b3bb-4b3d-9a6f-d5065f4356ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7847497073c46afbeaa601617115b06'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:18.491090 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1f2aa5ac-b3bb-4b3d-9a6f-d5065f4356ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7847497073c46afbeaa601617115b06'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:18.505325 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1f2aa5ac-b3bb-4b3d-9a6f-d5065f4356ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7847497073c46afbeaa601617115b06'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:18.520418 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1f2aa5ac-b3bb-4b3d-9a6f-d5065f4356ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7847497073c46afbeaa601617115b06'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:18.532383 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 901/1786] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:42:18 2022] DELETE /identity/v3/users/a7847497073c46afbeaa601617115b06 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:19.012884 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d43315f-b512-47d1-9713-39b71674f0d3 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:19.014376 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-1d43315f-b512-47d1-9713-39b71674f0d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:19.034923 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d43315f-b512-47d1-9713-39b71674f0d3 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:19.096393 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:19.118379 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b74f5-6573-4368-bf19-23f16963acb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:19.119572 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b74f5-6573-4368-bf19-23f16963acb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:19.119923 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b74f5-6573-4368-bf19-23f16963acb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:19.120261 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b74f5-6573-4368-bf19-23f16963acb2 admin admin] PATH_INFO: `/v3/projects/6a0e4b31069447ebb3cf0210cac307d6` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:19.122217 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7b74f5-6573-4368-bf19-23f16963acb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a0e4b31069447ebb3cf0210cac307d6)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:19.124466 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7b74f5-6573-4368-bf19-23f16963acb2 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:19.181226 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-cd7b74f5-6573-4368-bf19-23f16963acb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a0e4b31069447ebb3cf0210cac307d6', 'request_id': 'req-cd7b74f5-6573-4368-bf19-23f16963acb2'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:19.194272 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-cd7b74f5-6573-4368-bf19-23f16963acb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a0e4b31069447ebb3cf0210cac307d6', 'request_id': 'req-cd7b74f5-6573-4368-bf19-23f16963acb2'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:19.205918 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-cd7b74f5-6573-4368-bf19-23f16963acb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a0e4b31069447ebb3cf0210cac307d6', 'request_id': 'req-cd7b74f5-6573-4368-bf19-23f16963acb2'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:19.222288 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-cd7b74f5-6573-4368-bf19-23f16963acb2 admin admin] The token cache is being invalidate because project 6a0e4b31069447ebb3cf0210cac307d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:42:19.222715 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-cd7b74f5-6573-4368-bf19-23f16963acb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:19.223478 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-cd7b74f5-6573-4368-bf19-23f16963acb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:19.225721 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 886/1787] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:42:19 2022] DELETE /identity/v3/projects/6a0e4b31069447ebb3cf0210cac307d6 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:19.231258 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed15f23-348f-41d4-910c-d8f9c7cac37f None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:19.232540 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed15f23-348f-41d4-910c-d8f9c7cac37f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:19.233053 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed15f23-348f-41d4-910c-d8f9c7cac37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:19.260334 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-6ed15f23-348f-41d4-910c-d8f9c7cac37f None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:19.262375 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6ed15f23-348f-41d4-910c-d8f9c7cac37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:19.336852 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 902/1788] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:42:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:42:19.346603 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c74d9-f0c3-43af-9c0d-2e90b1e704d5 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:19.346753 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c74d9-f0c3-43af-9c0d-2e90b1e704d5 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:19.346919 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56c74d9-f0c3-43af-9c0d-2e90b1e704d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:19.382076 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-c56c74d9-f0c3-43af-9c0d-2e90b1e704d5 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:19.383858 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c56c74d9-f0c3-43af-9c0d-2e90b1e704d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:19.395753 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 887/1789] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:42:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:19.399688 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed15f23-348f-41d4-910c-d8f9c7cac37f None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:19.411451 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed15f23-348f-41d4-910c-d8f9c7cac37f None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:19.417747 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b08478-dae9-452c-a5f1-3c68ed7b75a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:19.418432 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b08478-dae9-452c-a5f1-3c68ed7b75a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:19.418718 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b08478-dae9-452c-a5f1-3c68ed7b75a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:19.418994 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b08478-dae9-452c-a5f1-3c68ed7b75a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:19.419757 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b08478-dae9-452c-a5f1-3c68ed7b75a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:42:19.419989 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b08478-dae9-452c-a5f1-3c68ed7b75a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:19.421892 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b08478-dae9-452c-a5f1-3c68ed7b75a5 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:19.451964 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56c74d9-f0c3-43af-9c0d-2e90b1e704d5 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:19.457602 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 903/1790] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:42:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:42:19.463461 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56c74d9-f0c3-43af-9c0d-2e90b1e704d5 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:19.470462 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd18819-8f41-42b8-a1cd-20d22d71e69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:19.474333 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd18819-8f41-42b8-a1cd-20d22d71e69f admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:19.474851 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd18819-8f41-42b8-a1cd-20d22d71e69f admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:19.475461 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd18819-8f41-42b8-a1cd-20d22d71e69f admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:19.491178 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd18819-8f41-42b8-a1cd-20d22d71e69f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:19.493397 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd18819-8f41-42b8-a1cd-20d22d71e69f admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:19.567298 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 888/1791] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:42:19 2022] POST /identity/v3/projects => generated 385 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:19.571396 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b08478-dae9-452c-a5f1-3c68ed7b75a5 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:19.581541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b08478-dae9-452c-a5f1-3c68ed7b75a5 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:19.623163 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:19.663563 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4089b0-4e5e-4458-9c4a-58365fd5a949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:19.664433 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4089b0-4e5e-4458-9c4a-58365fd5a949 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:19.664972 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4089b0-4e5e-4458-9c4a-58365fd5a949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:19.665248 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4089b0-4e5e-4458-9c4a-58365fd5a949 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:19.666029 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4089b0-4e5e-4458-9c4a-58365fd5a949 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:19.668378 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4089b0-4e5e-4458-9c4a-58365fd5a949 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:19.692385 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 904/1792] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:42:19 2022] POST /identity/v3/users => generated 293 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:19.696758 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd18819-8f41-42b8-a1cd-20d22d71e69f admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:19.724295 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd18819-8f41-42b8-a1cd-20d22d71e69f admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:19.751355 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5befab77-8f1d-46bf-9097-27bca46cbac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:19.763208 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5befab77-8f1d-46bf-9097-27bca46cbac6 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:19.763208 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5befab77-8f1d-46bf-9097-27bca46cbac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:19.763208 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5befab77-8f1d-46bf-9097-27bca46cbac6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:19.763208 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5befab77-8f1d-46bf-9097-27bca46cbac6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:19.763208 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5befab77-8f1d-46bf-9097-27bca46cbac6 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:19.782865 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 889/1793] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:19 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:19.786576 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4089b0-4e5e-4458-9c4a-58365fd5a949 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:19.797260 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4089b0-4e5e-4458-9c4a-58365fd5a949 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:19.811237 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd83b9a3-23bf-4e35-a33c-3f512d4f19d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:19.811973 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd83b9a3-23bf-4e35-a33c-3f512d4f19d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:19.812319 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd83b9a3-23bf-4e35-a33c-3f512d4f19d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:19.812666 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd83b9a3-23bf-4e35-a33c-3f512d4f19d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:19.813579 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd83b9a3-23bf-4e35-a33c-3f512d4f19d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:19.816583 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd83b9a3-23bf-4e35-a33c-3f512d4f19d2 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:19.831754 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 905/1794] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:19 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:19.836920 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5befab77-8f1d-46bf-9097-27bca46cbac6 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:19.848827 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5befab77-8f1d-46bf-9097-27bca46cbac6 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:19.855722 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7592730e-3262-42bf-bf9b-bf18092154c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:19.856707 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7592730e-3262-42bf-bf9b-bf18092154c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:19.856989 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7592730e-3262-42bf-bf9b-bf18092154c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:19.857260 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7592730e-3262-42bf-bf9b-bf18092154c5 admin admin] PATH_INFO: `/v3/projects/854a0f7c36334d78b3a638079f4a1e32/users/f5c6d93a7db2492b84b50c949d2bf809/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:19.873805 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd83b9a3-23bf-4e35-a33c-3f512d4f19d2 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:19.875817 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-cd83b9a3-23bf-4e35-a33c-3f512d4f19d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:19.884651 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7592730e-3262-42bf-bf9b-bf18092154c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=854a0f7c36334d78b3a638079f4a1e32, user_id=f5c6d93a7db2492b84b50c949d2bf809, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:19.887808 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7592730e-3262-42bf-bf9b-bf18092154c5 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:19.889825 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd83b9a3-23bf-4e35-a33c-3f512d4f19d2 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:19.895212 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2037866-dc11-429f-8d33-92e9578b7d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:19.896033 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2037866-dc11-429f-8d33-92e9578b7d8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:19.896372 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2037866-dc11-429f-8d33-92e9578b7d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:19.896763 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2037866-dc11-429f-8d33-92e9578b7d8d admin admin] PATH_INFO: `/v3/users/984dcaeea04744dfb8bb7e11017cc1c0` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:19.898136 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2037866-dc11-429f-8d33-92e9578b7d8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=984dcaeea04744dfb8bb7e11017cc1c0)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:19.900315 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2037866-dc11-429f-8d33-92e9578b7d8d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:19.904266 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 890/1795] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:19 2022] PUT /identity/v3/projects/854a0f7c36334d78b3a638079f4a1e32/users/f5c6d93a7db2492b84b50c949d2bf809/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:19.914631 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4558c6e-13b4-4413-b39e-25adcf7cb6d2 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:19.914945 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4558c6e-13b4-4413-b39e-25adcf7cb6d2 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:19.915180 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4558c6e-13b4-4413-b39e-25adcf7cb6d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:19.948279 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-a4558c6e-13b4-4413-b39e-25adcf7cb6d2 None None] MFA Rules not processed for user `f5c6d93a7db2492b84b50c949d2bf809`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:19.949776 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a4558c6e-13b4-4413-b39e-25adcf7cb6d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:19.963420 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c2037866-dc11-429f-8d33-92e9578b7d8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '984dcaeea04744dfb8bb7e11017cc1c0'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:19.964699 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c2037866-dc11-429f-8d33-92e9578b7d8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '984dcaeea04744dfb8bb7e11017cc1c0'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:19.978505 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c2037866-dc11-429f-8d33-92e9578b7d8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '984dcaeea04744dfb8bb7e11017cc1c0'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:19.986562 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c2037866-dc11-429f-8d33-92e9578b7d8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '984dcaeea04744dfb8bb7e11017cc1c0'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:19.993900 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c2037866-dc11-429f-8d33-92e9578b7d8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '984dcaeea04744dfb8bb7e11017cc1c0'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:20.003350 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 906/1796] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:42:19 2022] DELETE /identity/v3/users/984dcaeea04744dfb8bb7e11017cc1c0 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:20.004687 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91beb8ef-625b-417c-bbb3-7cee3c0c7e7d None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:20.004978 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91beb8ef-625b-417c-bbb3-7cee3c0c7e7d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:20.005250 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91beb8ef-625b-417c-bbb3-7cee3c0c7e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:20.025211 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-91beb8ef-625b-417c-bbb3-7cee3c0c7e7d None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:20.025513 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:20.026484 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-91beb8ef-625b-417c-bbb3-7cee3c0c7e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:20.033081 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 891/1797] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:42:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:42:20.037075 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4558c6e-13b4-4413-b39e-25adcf7cb6d2 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:20.041173 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a4558c6e-13b4-4413-b39e-25adcf7cb6d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:20.116858 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 907/1798] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:42:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:20.130335 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91beb8ef-625b-417c-bbb3-7cee3c0c7e7d None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:20.132132 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-91beb8ef-625b-417c-bbb3-7cee3c0c7e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:20.163468 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4558c6e-13b4-4413-b39e-25adcf7cb6d2 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:20.168935 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e18ca1-1367-46a0-9ead-7d01b7b1c85c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:20.169573 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e18ca1-1367-46a0-9ead-7d01b7b1c85c admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:20.169997 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e18ca1-1367-46a0-9ead-7d01b7b1c85c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:20.170271 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e18ca1-1367-46a0-9ead-7d01b7b1c85c admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:20.171080 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e18ca1-1367-46a0-9ead-7d01b7b1c85c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:20.173011 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e18ca1-1367-46a0-9ead-7d01b7b1c85c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:20.175454 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:20.224387 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 892/1799] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:42:20 2022] POST /identity/v3/projects => generated 387 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:42:20.229392 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e18ca1-1367-46a0-9ead-7d01b7b1c85c admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:20.236883 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91beb8ef-625b-417c-bbb3-7cee3c0c7e7d None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:20.309266 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec10bbbd-5c2d-46e3-b2aa-6cd6c17d2f54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:20.310069 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10bbbd-5c2d-46e3-b2aa-6cd6c17d2f54 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:20.310442 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10bbbd-5c2d-46e3-b2aa-6cd6c17d2f54 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:20.310806 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10bbbd-5c2d-46e3-b2aa-6cd6c17d2f54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:20.331266 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e18ca1-1367-46a0-9ead-7d01b7b1c85c admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:20.337941 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a728ea-baeb-4c40-8d95-af91b3638eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:20.338755 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a728ea-baeb-4c40-8d95-af91b3638eb0 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:20.339999 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a728ea-baeb-4c40-8d95-af91b3638eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:20.340512 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a728ea-baeb-4c40-8d95-af91b3638eb0 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:20.342064 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a728ea-baeb-4c40-8d95-af91b3638eb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:20.344199 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a728ea-baeb-4c40-8d95-af91b3638eb0 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:20.371119 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 893/1800] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:42:20 2022] POST /identity/v3/users => generated 300 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:20.376617 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a728ea-baeb-4c40-8d95-af91b3638eb0 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:20.380395 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec10bbbd-5c2d-46e3-b2aa-6cd6c17d2f54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:20.381868 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec10bbbd-5c2d-46e3-b2aa-6cd6c17d2f54 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:20.389231 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a728ea-baeb-4c40-8d95-af91b3638eb0 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:20.394057 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 908/1801] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:42:20.394645 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e14edb3-cec9-49c3-a1a4-3cc2f2f715da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:20.395359 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14edb3-cec9-49c3-a1a4-3cc2f2f715da admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:20.395707 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14edb3-cec9-49c3-a1a4-3cc2f2f715da admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:20.396042 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14edb3-cec9-49c3-a1a4-3cc2f2f715da admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:20.397527 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e14edb3-cec9-49c3-a1a4-3cc2f2f715da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:20.399231 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e14edb3-cec9-49c3-a1a4-3cc2f2f715da admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:20.411230 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 894/1802] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:42:20.415414 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec10bbbd-5c2d-46e3-b2aa-6cd6c17d2f54 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:20.430420 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec10bbbd-5c2d-46e3-b2aa-6cd6c17d2f54 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:20.438593 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3c5d80-f84b-488d-94bd-cd2fef9191de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:20.439318 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c5d80-f84b-488d-94bd-cd2fef9191de admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:20.440020 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c5d80-f84b-488d-94bd-cd2fef9191de admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:20.440328 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c5d80-f84b-488d-94bd-cd2fef9191de admin admin] PATH_INFO: `/v3/projects/7b2aa358c2b0460c9dcfce361227a9ce/users/1d85f64500874fb29b7fe19ad0b72179/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:20.465145 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3c5d80-f84b-488d-94bd-cd2fef9191de admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b2aa358c2b0460c9dcfce361227a9ce, user_id=1d85f64500874fb29b7fe19ad0b72179, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:20.468317 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3c5d80-f84b-488d-94bd-cd2fef9191de admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:20.482304 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 909/1803] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:20 2022] PUT /identity/v3/projects/7b2aa358c2b0460c9dcfce361227a9ce/users/1d85f64500874fb29b7fe19ad0b72179/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:42:20.486264 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e14edb3-cec9-49c3-a1a4-3cc2f2f715da admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:20.510552 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e14edb3-cec9-49c3-a1a4-3cc2f2f715da admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:20.644629 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4abc25e-1165-4ac9-9823-ccd687d5b777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:20.645328 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4abc25e-1165-4ac9-9823-ccd687d5b777 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:20.645700 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4abc25e-1165-4ac9-9823-ccd687d5b777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:20.646060 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4abc25e-1165-4ac9-9823-ccd687d5b777 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:20.646861 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4abc25e-1165-4ac9-9823-ccd687d5b777 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:20.648516 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4abc25e-1165-4ac9-9823-ccd687d5b777 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:20.651308 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:20.667327 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3c5d80-f84b-488d-94bd-cd2fef9191de admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:20.668520 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 895/1804] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:20 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:20.678980 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3c5d80-f84b-488d-94bd-cd2fef9191de admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:20.684889 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761d7b97-5efe-4b77-9205-3afd26b978a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:20.685628 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761d7b97-5efe-4b77-9205-3afd26b978a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:20.686039 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761d7b97-5efe-4b77-9205-3afd26b978a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:20.686381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761d7b97-5efe-4b77-9205-3afd26b978a2 admin admin] PATH_INFO: `/v3/projects/7b2aa358c2b0460c9dcfce361227a9ce/users/1d85f64500874fb29b7fe19ad0b72179/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:20.688546 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761d7b97-5efe-4b77-9205-3afd26b978a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b2aa358c2b0460c9dcfce361227a9ce, user_id=1d85f64500874fb29b7fe19ad0b72179, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:20.691304 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761d7b97-5efe-4b77-9205-3afd26b978a2 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:20.703168 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 910/1805] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:20 2022] PUT /identity/v3/projects/7b2aa358c2b0460c9dcfce361227a9ce/users/1d85f64500874fb29b7fe19ad0b72179/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:20.714429 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175bee18-5e91-4187-85a2-423da70f52a9 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:20.714704 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175bee18-5e91-4187-85a2-423da70f52a9 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:20.714972 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175bee18-5e91-4187-85a2-423da70f52a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:20.748707 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-175bee18-5e91-4187-85a2-423da70f52a9 None None] MFA Rules not processed for user `1d85f64500874fb29b7fe19ad0b72179`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:20.750520 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-175bee18-5e91-4187-85a2-423da70f52a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:20.839917 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 896/1806] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:42:20 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:20.852738 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761d7b97-5efe-4b77-9205-3afd26b978a2 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:20.855900 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-761d7b97-5efe-4b77-9205-3afd26b978a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:20.948772 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:20.954240 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761d7b97-5efe-4b77-9205-3afd26b978a2 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:20.959176 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9affd30-9068-40c8-9159-011d3bfbfc0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:20.959885 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9affd30-9068-40c8-9159-011d3bfbfc0b service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:20.960164 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9affd30-9068-40c8-9159-011d3bfbfc0b service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:20.960506 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9affd30-9068-40c8-9159-011d3bfbfc0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:20.971794 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9affd30-9068-40c8-9159-011d3bfbfc0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:20.973418 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9affd30-9068-40c8-9159-011d3bfbfc0b service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:20.983812 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 911/1807] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:20 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:42:21.015343 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175bee18-5e91-4187-85a2-423da70f52a9 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:21.073629 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9affd30-9068-40c8-9159-011d3bfbfc0b service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:21.075357 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c9affd30-9068-40c8-9159-011d3bfbfc0b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:21.089955 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175bee18-5e91-4187-85a2-423da70f52a9 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:21.095414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06a2f07-5432-44e1-b090-400a197992ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:21.096321 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a2f07-5432-44e1-b090-400a197992ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:21.096698 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a2f07-5432-44e1-b090-400a197992ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:21.097069 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a2f07-5432-44e1-b090-400a197992ee admin admin] PATH_INFO: `/v3/users/f5c6d93a7db2492b84b50c949d2bf809` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:21.098441 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06a2f07-5432-44e1-b090-400a197992ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5c6d93a7db2492b84b50c949d2bf809)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:21.100481 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06a2f07-5432-44e1-b090-400a197992ee admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:21.156488 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9affd30-9068-40c8-9159-011d3bfbfc0b service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:21.162718 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eae59b6-ef01-4164-a9ca-9e8a19e0214e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:21.163438 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae59b6-ef01-4164-a9ca-9e8a19e0214e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:21.164060 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae59b6-ef01-4164-a9ca-9e8a19e0214e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:21.166130 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae59b6-ef01-4164-a9ca-9e8a19e0214e admin admin] PATH_INFO: `/v3/projects/3db0be6d2ea24d41af8363cacc9122f8` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:21.168068 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eae59b6-ef01-4164-a9ca-9e8a19e0214e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3db0be6d2ea24d41af8363cacc9122f8)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:21.170456 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eae59b6-ef01-4164-a9ca-9e8a19e0214e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:21.182427 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c06a2f07-5432-44e1-b090-400a197992ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5c6d93a7db2492b84b50c949d2bf809'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:21.191030 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c06a2f07-5432-44e1-b090-400a197992ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5c6d93a7db2492b84b50c949d2bf809'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:21.191900 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c06a2f07-5432-44e1-b090-400a197992ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5c6d93a7db2492b84b50c949d2bf809'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:21.202246 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c06a2f07-5432-44e1-b090-400a197992ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5c6d93a7db2492b84b50c949d2bf809'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:21.202917 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c06a2f07-5432-44e1-b090-400a197992ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5c6d93a7db2492b84b50c949d2bf809'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:21.232230 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9eae59b6-ef01-4164-a9ca-9e8a19e0214e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3db0be6d2ea24d41af8363cacc9122f8', 'request_id': 'req-9eae59b6-ef01-4164-a9ca-9e8a19e0214e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:21.234642 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 897/1808] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:42:21 2022] DELETE /identity/v3/users/f5c6d93a7db2492b84b50c949d2bf809 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:21.238948 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6295fd-4801-4790-a07e-4ec1b26232c2 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:21.239079 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9eae59b6-ef01-4164-a9ca-9e8a19e0214e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3db0be6d2ea24d41af8363cacc9122f8', 'request_id': 'req-9eae59b6-ef01-4164-a9ca-9e8a19e0214e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:21.239483 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6295fd-4801-4790-a07e-4ec1b26232c2 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:21.239734 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6295fd-4801-4790-a07e-4ec1b26232c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:21.246894 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9eae59b6-ef01-4164-a9ca-9e8a19e0214e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3db0be6d2ea24d41af8363cacc9122f8', 'request_id': 'req-9eae59b6-ef01-4164-a9ca-9e8a19e0214e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:21.247142 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:21.258806 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9eae59b6-ef01-4164-a9ca-9e8a19e0214e admin admin] The token cache is being invalidate because project 3db0be6d2ea24d41af8363cacc9122f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:42:21.259088 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9eae59b6-ef01-4164-a9ca-9e8a19e0214e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:21.259880 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9eae59b6-ef01-4164-a9ca-9e8a19e0214e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:21.261506 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 912/1809] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:42:21 2022] DELETE /identity/v3/projects/3db0be6d2ea24d41af8363cacc9122f8 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:21.265348 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eae59b6-ef01-4164-a9ca-9e8a19e0214e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:21.266296 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-ee6295fd-4801-4790-a07e-4ec1b26232c2 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:21.266774 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9eae59b6-ef01-4164-a9ca-9e8a19e0214e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:21.267893 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ee6295fd-4801-4790-a07e-4ec1b26232c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:21.341335 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 898/1810] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:42:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:42:21.341512 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eae59b6-ef01-4164-a9ca-9e8a19e0214e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:21.351090 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6295fd-4801-4790-a07e-4ec1b26232c2 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:21.352983 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ee6295fd-4801-4790-a07e-4ec1b26232c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:21.358120 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e66d1c9-19e0-4cb0-9cb4-65f96156fc8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:21.358785 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e66d1c9-19e0-4cb0-9cb4-65f96156fc8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:21.359070 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e66d1c9-19e0-4cb0-9cb4-65f96156fc8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:21.359350 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e66d1c9-19e0-4cb0-9cb4-65f96156fc8d admin admin] PATH_INFO: `/v3/users/47e562316d994f1987e5bef47eec3620` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:21.360719 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e66d1c9-19e0-4cb0-9cb4-65f96156fc8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=47e562316d994f1987e5bef47eec3620)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:21.363649 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e66d1c9-19e0-4cb0-9cb4-65f96156fc8d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:21.390686 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:21.422754 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6295fd-4801-4790-a07e-4ec1b26232c2 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:21.424133 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5e66d1c9-19e0-4cb0-9cb4-65f96156fc8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47e562316d994f1987e5bef47eec3620'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:21.425229 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5e66d1c9-19e0-4cb0-9cb4-65f96156fc8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47e562316d994f1987e5bef47eec3620'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:21.426818 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d89260-0a23-4b43-aa20-f2a58741e608 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:21.427550 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d89260-0a23-4b43-aa20-f2a58741e608 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:21.427898 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d89260-0a23-4b43-aa20-f2a58741e608 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:21.428239 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d89260-0a23-4b43-aa20-f2a58741e608 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:21.439607 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5e66d1c9-19e0-4cb0-9cb4-65f96156fc8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47e562316d994f1987e5bef47eec3620'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:21.449400 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d89260-0a23-4b43-aa20-f2a58741e608 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:21.451302 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d89260-0a23-4b43-aa20-f2a58741e608 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:21.452583 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5e66d1c9-19e0-4cb0-9cb4-65f96156fc8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47e562316d994f1987e5bef47eec3620'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:21.464565 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5e66d1c9-19e0-4cb0-9cb4-65f96156fc8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47e562316d994f1987e5bef47eec3620'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:21.472590 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 913/1811] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:42:21 2022] DELETE /identity/v3/users/47e562316d994f1987e5bef47eec3620 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:42:21.473053 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 899/1812] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:42:21.995669 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e66d1c9-19e0-4cb0-9cb4-65f96156fc8d admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:21.997398 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5e66d1c9-19e0-4cb0-9cb4-65f96156fc8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:22.013633 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d89260-0a23-4b43-aa20-f2a58741e608 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:22.027510 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d89260-0a23-4b43-aa20-f2a58741e608 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:22.112290 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8b078d-3ef8-48c2-97d7-18226f3af75b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:22.112961 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b078d-3ef8-48c2-97d7-18226f3af75b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:22.113202 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b078d-3ef8-48c2-97d7-18226f3af75b admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:22.113635 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b078d-3ef8-48c2-97d7-18226f3af75b admin admin] PATH_INFO: `/v3/projects/f871eb2d28c340b186085cb0032947fe` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:22.114854 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8b078d-3ef8-48c2-97d7-18226f3af75b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f871eb2d28c340b186085cb0032947fe)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:22.115871 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e66d1c9-19e0-4cb0-9cb4-65f96156fc8d admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:22.116770 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8b078d-3ef8-48c2-97d7-18226f3af75b admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:22.121005 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cc7e5b-1985-4dbe-8156-3cbafe89babc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:22.121645 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc7e5b-1985-4dbe-8156-3cbafe89babc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:22.121961 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc7e5b-1985-4dbe-8156-3cbafe89babc admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:22.122242 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc7e5b-1985-4dbe-8156-3cbafe89babc admin admin] PATH_INFO: `/v3/projects/854a0f7c36334d78b3a638079f4a1e32` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:22.123452 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cc7e5b-1985-4dbe-8156-3cbafe89babc admin admin] RBAC: Authorizing `identity:delete_project(project_id=854a0f7c36334d78b3a638079f4a1e32)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:22.125659 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cc7e5b-1985-4dbe-8156-3cbafe89babc admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:22.183648 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:22.184916 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-76cc7e5b-1985-4dbe-8156-3cbafe89babc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '854a0f7c36334d78b3a638079f4a1e32', 'request_id': 'req-76cc7e5b-1985-4dbe-8156-3cbafe89babc'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:22.195986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ec8b078d-3ef8-48c2-97d7-18226f3af75b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f871eb2d28c340b186085cb0032947fe', 'request_id': 'req-ec8b078d-3ef8-48c2-97d7-18226f3af75b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:22.202962 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-76cc7e5b-1985-4dbe-8156-3cbafe89babc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '854a0f7c36334d78b3a638079f4a1e32', 'request_id': 'req-76cc7e5b-1985-4dbe-8156-3cbafe89babc'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:22.207647 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ec8b078d-3ef8-48c2-97d7-18226f3af75b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f871eb2d28c340b186085cb0032947fe', 'request_id': 'req-ec8b078d-3ef8-48c2-97d7-18226f3af75b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:22.213786 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-76cc7e5b-1985-4dbe-8156-3cbafe89babc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '854a0f7c36334d78b3a638079f4a1e32', 'request_id': 'req-76cc7e5b-1985-4dbe-8156-3cbafe89babc'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:22.215513 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ec8b078d-3ef8-48c2-97d7-18226f3af75b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f871eb2d28c340b186085cb0032947fe', 'request_id': 'req-ec8b078d-3ef8-48c2-97d7-18226f3af75b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:22.226294 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ec8b078d-3ef8-48c2-97d7-18226f3af75b admin admin] The token cache is being invalidate because project f871eb2d28c340b186085cb0032947fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:42:22.226587 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ec8b078d-3ef8-48c2-97d7-18226f3af75b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:22.227257 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ec8b078d-3ef8-48c2-97d7-18226f3af75b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:22.228840 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 900/1813] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:42:22 2022] DELETE /identity/v3/projects/f871eb2d28c340b186085cb0032947fe => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:42:22.233348 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-76cc7e5b-1985-4dbe-8156-3cbafe89babc admin admin] The token cache is being invalidate because project 854a0f7c36334d78b3a638079f4a1e32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:42:22.233652 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-76cc7e5b-1985-4dbe-8156-3cbafe89babc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:22.234164 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0807de0f-68f6-4b4d-99b1-3f2915939eef None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:22.234401 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-76cc7e5b-1985-4dbe-8156-3cbafe89babc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:22.234574 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0807de0f-68f6-4b4d-99b1-3f2915939eef None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:22.234909 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0807de0f-68f6-4b4d-99b1-3f2915939eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:22.236009 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 914/1814] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:42:21 2022] DELETE /identity/v3/projects/854a0f7c36334d78b3a638079f4a1e32 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:22.239895 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cc7e5b-1985-4dbe-8156-3cbafe89babc admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:22.241310 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-76cc7e5b-1985-4dbe-8156-3cbafe89babc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:22.269647 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-0807de0f-68f6-4b4d-99b1-3f2915939eef None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:22.276037 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-0807de0f-68f6-4b4d-99b1-3f2915939eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:22.336344 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:22.370153 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 901/1815] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:42:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:22.378415 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cc7e5b-1985-4dbe-8156-3cbafe89babc admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:22.378586 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ea30c-111e-4606-88b3-11f3ee095314 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:22.378841 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ea30c-111e-4606-88b3-11f3ee095314 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:22.379118 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ea30c-111e-4606-88b3-11f3ee095314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:22.384154 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9d36c8-b684-4398-9b0b-cd175e48668b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:22.385145 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d36c8-b684-4398-9b0b-cd175e48668b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:22.385571 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d36c8-b684-4398-9b0b-cd175e48668b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:22.386101 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d36c8-b684-4398-9b0b-cd175e48668b admin admin] PATH_INFO: `/v3/users/1d85f64500874fb29b7fe19ad0b72179` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:22.387409 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9d36c8-b684-4398-9b0b-cd175e48668b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d85f64500874fb29b7fe19ad0b72179)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:22.389625 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9d36c8-b684-4398-9b0b-cd175e48668b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:22.407463 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-b77ea30c-111e-4606-88b3-11f3ee095314 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:22.407463 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b77ea30c-111e-4606-88b3-11f3ee095314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:22.421996 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 902/1816] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:42:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:22.423066 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77ea30c-111e-4606-88b3-11f3ee095314 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:22.442321 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77ea30c-111e-4606-88b3-11f3ee095314 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:22.448574 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75013a16-fbb5-49b0-b1c3-a6aa31b28ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:22.449334 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75013a16-fbb5-49b0-b1c3-a6aa31b28ad4 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:22.450122 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75013a16-fbb5-49b0-b1c3-a6aa31b28ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:22.450499 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75013a16-fbb5-49b0-b1c3-a6aa31b28ad4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:22.451242 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75013a16-fbb5-49b0-b1c3-a6aa31b28ad4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:42:22.451703 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75013a16-fbb5-49b0-b1c3-a6aa31b28ad4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:22.454795 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75013a16-fbb5-49b0-b1c3-a6aa31b28ad4 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:22.462061 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-bb9d36c8-b684-4398-9b0b-cd175e48668b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d85f64500874fb29b7fe19ad0b72179'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:22.462958 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-bb9d36c8-b684-4398-9b0b-cd175e48668b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d85f64500874fb29b7fe19ad0b72179'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:22.476345 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 903/1817] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:42:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:42:22.478563 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75013a16-fbb5-49b0-b1c3-a6aa31b28ad4 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:22.493894 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-bb9d36c8-b684-4398-9b0b-cd175e48668b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d85f64500874fb29b7fe19ad0b72179'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:22.500431 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75013a16-fbb5-49b0-b1c3-a6aa31b28ad4 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:22.507886 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d676395-abad-4e84-a49e-ced930ed997d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:22.508633 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d676395-abad-4e84-a49e-ced930ed997d admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:22.509587 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d676395-abad-4e84-a49e-ced930ed997d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:22.510236 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d676395-abad-4e84-a49e-ced930ed997d admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:22.511533 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d676395-abad-4e84-a49e-ced930ed997d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:22.511691 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-bb9d36c8-b684-4398-9b0b-cd175e48668b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d85f64500874fb29b7fe19ad0b72179'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:22.513500 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d676395-abad-4e84-a49e-ced930ed997d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:22.520268 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-bb9d36c8-b684-4398-9b0b-cd175e48668b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d85f64500874fb29b7fe19ad0b72179'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:22.536184 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 915/1818] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:42:22 2022] DELETE /identity/v3/users/1d85f64500874fb29b7fe19ad0b72179 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:22.553912 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 904/1819] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:42:22 2022] POST /identity/v3/projects => generated 365 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:22.554657 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9d36c8-b684-4398-9b0b-cd175e48668b admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:22.558535 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-bb9d36c8-b684-4398-9b0b-cd175e48668b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:22.723483 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9d36c8-b684-4398-9b0b-cd175e48668b admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:22.803658 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442251b-aca2-446c-97f4-f20c29b08dc9 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:22.804856 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442251b-aca2-446c-97f4-f20c29b08dc9 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:22.805285 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442251b-aca2-446c-97f4-f20c29b08dc9 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:22.807304 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 905/1820] 10.222.0.52 () {54 vars in 921 bytes} [Wed Sep 28 15:42:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:42:22.807890 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:22.820989 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911fca04-ba4c-48e6-9493-36dc104b564e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:22.822800 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911fca04-ba4c-48e6-9493-36dc104b564e admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:22.822800 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911fca04-ba4c-48e6-9493-36dc104b564e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:22.822800 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911fca04-ba4c-48e6-9493-36dc104b564e admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:22.824323 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911fca04-ba4c-48e6-9493-36dc104b564e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:22.825626 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911fca04-ba4c-48e6-9493-36dc104b564e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:22.831235 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f004732-36e8-40b6-bb6b-10be71714363 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:22.834362 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f004732-36e8-40b6-bb6b-10be71714363 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:22.836985 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f004732-36e8-40b6-bb6b-10be71714363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:22.873231 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 916/1821] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:42:22 2022] POST /identity/v3/users => generated 283 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:42:22.875421 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911fca04-ba4c-48e6-9493-36dc104b564e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:22.890067 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-2f004732-36e8-40b6-bb6b-10be71714363 None None] MFA Rules not processed for user `8daf14b64bf04b2ba74d2fe1fd41ff71`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:22.891849 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-2f004732-36e8-40b6-bb6b-10be71714363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:22.898740 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911fca04-ba4c-48e6-9493-36dc104b564e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:22.905785 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d1b0f-b81d-45d6-b8b4-23d69ed10d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:22.906835 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d1b0f-b81d-45d6-b8b4-23d69ed10d6d admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:22.907121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d1b0f-b81d-45d6-b8b4-23d69ed10d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:22.907401 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d1b0f-b81d-45d6-b8b4-23d69ed10d6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:22.908271 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9d1b0f-b81d-45d6-b8b4-23d69ed10d6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:22.910193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9d1b0f-b81d-45d6-b8b4-23d69ed10d6d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:22.929971 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 917/1822] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:22 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:22.931615 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d1b0f-b81d-45d6-b8b4-23d69ed10d6d admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:22.947389 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d1b0f-b81d-45d6-b8b4-23d69ed10d6d admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:22.955382 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84807f53-bced-4ff3-bbaf-1bf7f8ee8d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:22.955616 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84807f53-bced-4ff3-bbaf-1bf7f8ee8d38 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:22.955910 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84807f53-bced-4ff3-bbaf-1bf7f8ee8d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:22.956188 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84807f53-bced-4ff3-bbaf-1bf7f8ee8d38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:22.956870 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84807f53-bced-4ff3-bbaf-1bf7f8ee8d38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:22.958801 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84807f53-bced-4ff3-bbaf-1bf7f8ee8d38 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:22.974649 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 918/1823] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:22 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:22.978967 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84807f53-bced-4ff3-bbaf-1bf7f8ee8d38 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:22.995027 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84807f53-bced-4ff3-bbaf-1bf7f8ee8d38 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:23.001089 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d52dd57-4df3-4d92-bf79-f9d95b00b228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:23.002011 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d52dd57-4df3-4d92-bf79-f9d95b00b228 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:23.002401 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d52dd57-4df3-4d92-bf79-f9d95b00b228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:23.002678 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d52dd57-4df3-4d92-bf79-f9d95b00b228 admin admin] PATH_INFO: `/v3/projects/edddf0c9177b46cf9ca89f87543a4e13/users/207922b68d704a18bc4fe64de88775f0/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:23.006221 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 906/1824] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Sep 28 15:42:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:23.023783 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d52dd57-4df3-4d92-bf79-f9d95b00b228 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edddf0c9177b46cf9ca89f87543a4e13, user_id=207922b68d704a18bc4fe64de88775f0, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:23.028648 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d52dd57-4df3-4d92-bf79-f9d95b00b228 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:23.042144 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f004732-36e8-40b6-bb6b-10be71714363 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:23.043678 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-2f004732-36e8-40b6-bb6b-10be71714363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:23.061188 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 919/1825] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:22 2022] PUT /identity/v3/projects/edddf0c9177b46cf9ca89f87543a4e13/users/207922b68d704a18bc4fe64de88775f0/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:23.064069 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8da0d3-ed43-47b1-936b-ac8a622d2802 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:23.066406 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8da0d3-ed43-47b1-936b-ac8a622d2802 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:23.066406 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8da0d3-ed43-47b1-936b-ac8a622d2802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:23.075602 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f004732-36e8-40b6-bb6b-10be71714363 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:23.148049 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:23.154187 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-cd8da0d3-ed43-47b1-936b-ac8a622d2802 None None] MFA Rules not processed for user `207922b68d704a18bc4fe64de88775f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:23.157333 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-cd8da0d3-ed43-47b1-936b-ac8a622d2802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:23.244922 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6384c225-8ee2-4607-8378-0d70bfa0953c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:23.246188 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384c225-8ee2-4607-8378-0d70bfa0953c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:23.246408 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384c225-8ee2-4607-8378-0d70bfa0953c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:23.246681 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384c225-8ee2-4607-8378-0d70bfa0953c admin admin] PATH_INFO: `/v3/projects/7b2aa358c2b0460c9dcfce361227a9ce` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:23.248232 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6384c225-8ee2-4607-8378-0d70bfa0953c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b2aa358c2b0460c9dcfce361227a9ce)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:23.252321 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6384c225-8ee2-4607-8378-0d70bfa0953c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:23.283785 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 920/1826] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:42:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:23.285282 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8da0d3-ed43-47b1-936b-ac8a622d2802 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:23.287069 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-cd8da0d3-ed43-47b1-936b-ac8a622d2802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:23.358867 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-6384c225-8ee2-4607-8378-0d70bfa0953c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b2aa358c2b0460c9dcfce361227a9ce', 'request_id': 'req-6384c225-8ee2-4607-8378-0d70bfa0953c'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:23.375648 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-6384c225-8ee2-4607-8378-0d70bfa0953c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b2aa358c2b0460c9dcfce361227a9ce', 'request_id': 'req-6384c225-8ee2-4607-8378-0d70bfa0953c'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:23.384305 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8da0d3-ed43-47b1-936b-ac8a622d2802 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:23.386717 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-6384c225-8ee2-4607-8378-0d70bfa0953c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b2aa358c2b0460c9dcfce361227a9ce', 'request_id': 'req-6384c225-8ee2-4607-8378-0d70bfa0953c'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:23.400152 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-6384c225-8ee2-4607-8378-0d70bfa0953c admin admin] The token cache is being invalidate because project 7b2aa358c2b0460c9dcfce361227a9ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:42:23.400857 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-6384c225-8ee2-4607-8378-0d70bfa0953c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:23.402245 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-6384c225-8ee2-4607-8378-0d70bfa0953c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:23.405275 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 907/1827] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:42:23 2022] DELETE /identity/v3/projects/7b2aa358c2b0460c9dcfce361227a9ce => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:42:23.406367 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6384c225-8ee2-4607-8378-0d70bfa0953c admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:23.408106 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-6384c225-8ee2-4607-8378-0d70bfa0953c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:23.425905 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:23.478362 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7ff2f0-e913-4508-a961-efdc033eaf77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf14b64bf04b2ba74d2fe1fd41ff71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:23.479317 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ff2f0-e913-4508-a961-efdc033eaf77 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:23.480309 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ff2f0-e913-4508-a961-efdc033eaf77 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:23.481045 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ff2f0-e913-4508-a961-efdc033eaf77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:23.483825 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5f7ff2f0-e913-4508-a961-efdc033eaf77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:23.498914 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7ff2f0-e913-4508-a961-efdc033eaf77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:23.501018 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6384c225-8ee2-4607-8378-0d70bfa0953c admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:23.502123 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7ff2f0-e913-4508-a961-efdc033eaf77 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:23.507017 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7f8d30-21eb-4cfb-898e-fc00518dcb1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:23.507775 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f8d30-21eb-4cfb-898e-fc00518dcb1b admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:23.508116 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f8d30-21eb-4cfb-898e-fc00518dcb1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:23.508450 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f8d30-21eb-4cfb-898e-fc00518dcb1b admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:23.509152 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7f8d30-21eb-4cfb-898e-fc00518dcb1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:23.511522 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7f8d30-21eb-4cfb-898e-fc00518dcb1b admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:23.531971 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 921/1828] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:42:23.533282 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25cb137-d7cc-4ea4-8b35-9e56ab55d6eb None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:23.533573 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25cb137-d7cc-4ea4-8b35-9e56ab55d6eb None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:23.533879 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25cb137-d7cc-4ea4-8b35-9e56ab55d6eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:23.555093 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 908/1829] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:42:23 2022] POST /identity/v3/projects => generated 363 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:23.560535 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7f8d30-21eb-4cfb-898e-fc00518dcb1b admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:23.597325 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-c25cb137-d7cc-4ea4-8b35-9e56ab55d6eb None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:23.599211 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c25cb137-d7cc-4ea4-8b35-9e56ab55d6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:23.633575 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:23.682908 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7f8d30-21eb-4cfb-898e-fc00518dcb1b admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:23.695888 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c7d0dc-de5c-4c72-9738-dba39b000a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:23.699028 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c7d0dc-de5c-4c72-9738-dba39b000a13 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:23.699405 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c7d0dc-de5c-4c72-9738-dba39b000a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:23.699741 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c7d0dc-de5c-4c72-9738-dba39b000a13 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:23.700459 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c7d0dc-de5c-4c72-9738-dba39b000a13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:23.705436 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c7d0dc-de5c-4c72-9738-dba39b000a13 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:23.745888 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 922/1830] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:42:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:23.747520 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 909/1831] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:42:23 2022] POST /identity/v3/users => generated 288 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:42:23.751894 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f53dedc-68cf-439a-bc59-c488c5f91efa None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:23.753402 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f53dedc-68cf-439a-bc59-c488c5f91efa None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:23.753751 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f53dedc-68cf-439a-bc59-c488c5f91efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:23.754503 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c7d0dc-de5c-4c72-9738-dba39b000a13 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:23.784758 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c7d0dc-de5c-4c72-9738-dba39b000a13 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:23.787145 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-0f53dedc-68cf-439a-bc59-c488c5f91efa None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:23.789100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0f53dedc-68cf-439a-bc59-c488c5f91efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:23.790729 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2da8453-dc34-4202-b36c-36a3485d4849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:23.794034 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da8453-dc34-4202-b36c-36a3485d4849 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:23.794602 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da8453-dc34-4202-b36c-36a3485d4849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:23.795055 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da8453-dc34-4202-b36c-36a3485d4849 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:23.796004 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2da8453-dc34-4202-b36c-36a3485d4849 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:23.797986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2da8453-dc34-4202-b36c-36a3485d4849 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:23.801105 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 923/1832] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:42:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:23.808438 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f53dedc-68cf-439a-bc59-c488c5f91efa None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:23.819036 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 910/1833] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:23 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:23.820518 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f53dedc-68cf-439a-bc59-c488c5f91efa None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:23.823615 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2da8453-dc34-4202-b36c-36a3485d4849 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:23.828976 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5769217-7ad2-4c36-92b8-1a5017d7f0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:23.829985 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5769217-7ad2-4c36-92b8-1a5017d7f0b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:23.830380 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5769217-7ad2-4c36-92b8-1a5017d7f0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:23.830758 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5769217-7ad2-4c36-92b8-1a5017d7f0b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:23.831809 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5769217-7ad2-4c36-92b8-1a5017d7f0b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:42:23.832237 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5769217-7ad2-4c36-92b8-1a5017d7f0b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:23.835180 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5769217-7ad2-4c36-92b8-1a5017d7f0b2 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:23.837167 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2da8453-dc34-4202-b36c-36a3485d4849 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:23.844188 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a40eca-4b5c-416f-be2a-890d895e21bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:23.845174 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a40eca-4b5c-416f-be2a-890d895e21bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:23.845892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a40eca-4b5c-416f-be2a-890d895e21bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:23.846329 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a40eca-4b5c-416f-be2a-890d895e21bd admin admin] PATH_INFO: `/v3/projects/eb0582b461b44e1b84628f5c18e12483/users/ca5944ec970446068f9149e7b8e09443/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:23.858960 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 924/1834] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:42:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:42:23.860227 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5769217-7ad2-4c36-92b8-1a5017d7f0b2 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:23.872469 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5769217-7ad2-4c36-92b8-1a5017d7f0b2 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:23.880091 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cb3c26-41d6-4f29-ac16-d2c01545a472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:23.880965 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb3c26-41d6-4f29-ac16-d2c01545a472 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:23.881669 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb3c26-41d6-4f29-ac16-d2c01545a472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:23.881798 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb3c26-41d6-4f29-ac16-d2c01545a472 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:23.882507 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cb3c26-41d6-4f29-ac16-d2c01545a472 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:23.883085 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a40eca-4b5c-416f-be2a-890d895e21bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb0582b461b44e1b84628f5c18e12483, user_id=ca5944ec970446068f9149e7b8e09443, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:23.884680 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cb3c26-41d6-4f29-ac16-d2c01545a472 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:23.889284 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a40eca-4b5c-416f-be2a-890d895e21bd admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:23.922121 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 911/1835] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:23 2022] PUT /identity/v3/projects/eb0582b461b44e1b84628f5c18e12483/users/ca5944ec970446068f9149e7b8e09443/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:23.931617 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a40eca-4b5c-416f-be2a-890d895e21bd admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:23.951817 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 925/1836] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:42:23 2022] POST /identity/v3/projects => generated 381 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:23.955660 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cb3c26-41d6-4f29-ac16-d2c01545a472 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:24.025333 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a40eca-4b5c-416f-be2a-890d895e21bd admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:24.054347 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cb3c26-41d6-4f29-ac16-d2c01545a472 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:24.060596 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad549f2a-2a28-41b1-93cd-fae6815c0726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:24.061428 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad549f2a-2a28-41b1-93cd-fae6815c0726 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:24.061780 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad549f2a-2a28-41b1-93cd-fae6815c0726 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:24.062171 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad549f2a-2a28-41b1-93cd-fae6815c0726 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:24.062854 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad549f2a-2a28-41b1-93cd-fae6815c0726 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:24.064968 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad549f2a-2a28-41b1-93cd-fae6815c0726 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:24.097897 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 926/1837] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:42:23 2022] POST /identity/v3/users => generated 291 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:24.099821 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad549f2a-2a28-41b1-93cd-fae6815c0726 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:24.112211 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:24.119895 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad549f2a-2a28-41b1-93cd-fae6815c0726 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:24.124192 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6629c048-d56b-41e4-b51a-a600dcaa648b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:24.124834 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629c048-d56b-41e4-b51a-a600dcaa648b admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:24.125312 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629c048-d56b-41e4-b51a-a600dcaa648b admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:24.125661 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629c048-d56b-41e4-b51a-a600dcaa648b admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:24.126468 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6629c048-d56b-41e4-b51a-a600dcaa648b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:24.128202 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6629c048-d56b-41e4-b51a-a600dcaa648b admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:24.130225 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8396811c-b288-48ba-ad81-f92a17cbb8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:24.131203 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396811c-b288-48ba-ad81-f92a17cbb8da admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:24.131681 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396811c-b288-48ba-ad81-f92a17cbb8da admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:24.132138 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396811c-b288-48ba-ad81-f92a17cbb8da admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:24.133215 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8396811c-b288-48ba-ad81-f92a17cbb8da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:24.136440 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8396811c-b288-48ba-ad81-f92a17cbb8da admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:24.140603 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 912/1838] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:23 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:24.146373 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6629c048-d56b-41e4-b51a-a600dcaa648b admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:24.154884 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 927/1839] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:24 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:42:24.158903 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8396811c-b288-48ba-ad81-f92a17cbb8da admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:24.165688 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6629c048-d56b-41e4-b51a-a600dcaa648b admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:24.173536 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c63433d-0b1c-458d-aa53-8efe1ff6f1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:24.174637 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63433d-0b1c-458d-aa53-8efe1ff6f1f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:24.175307 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63433d-0b1c-458d-aa53-8efe1ff6f1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:24.175433 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8396811c-b288-48ba-ad81-f92a17cbb8da admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:24.175763 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63433d-0b1c-458d-aa53-8efe1ff6f1f4 admin admin] PATH_INFO: `/v3/projects/eb0582b461b44e1b84628f5c18e12483/users/ca5944ec970446068f9149e7b8e09443/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:24.180117 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c63433d-0b1c-458d-aa53-8efe1ff6f1f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb0582b461b44e1b84628f5c18e12483, user_id=ca5944ec970446068f9149e7b8e09443, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:24.182127 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c63433d-0b1c-458d-aa53-8efe1ff6f1f4 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:24.183629 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef4917e-ef3f-4899-a04e-268709cd64e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:24.184422 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef4917e-ef3f-4899-a04e-268709cd64e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:24.184856 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef4917e-ef3f-4899-a04e-268709cd64e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:24.185326 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef4917e-ef3f-4899-a04e-268709cd64e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:24.186792 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef4917e-ef3f-4899-a04e-268709cd64e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:24.189426 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef4917e-ef3f-4899-a04e-268709cd64e8 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:24.202393 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 928/1840] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:24 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:24.206144 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef4917e-ef3f-4899-a04e-268709cd64e8 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:24.211378 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 913/1841] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:24 2022] PUT /identity/v3/projects/eb0582b461b44e1b84628f5c18e12483/users/ca5944ec970446068f9149e7b8e09443/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:42:24.222923 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db24f9-ee3e-4af9-abc1-65a3cb70ad35 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:24.223212 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db24f9-ee3e-4af9-abc1-65a3cb70ad35 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:24.223489 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db24f9-ee3e-4af9-abc1-65a3cb70ad35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:24.226112 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef4917e-ef3f-4899-a04e-268709cd64e8 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:24.258000 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-e7db24f9-ee3e-4af9-abc1-65a3cb70ad35 None None] MFA Rules not processed for user `ca5944ec970446068f9149e7b8e09443`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:24.259217 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e7db24f9-ee3e-4af9-abc1-65a3cb70ad35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:24.278803 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:24.328563 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1740e5-2812-478b-b88a-dea4314ffb5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:24.328563 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1740e5-2812-478b-b88a-dea4314ffb5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:24.329073 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1740e5-2812-478b-b88a-dea4314ffb5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:24.329073 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1740e5-2812-478b-b88a-dea4314ffb5b admin admin] PATH_INFO: `/v3/projects/0b8f5d7ef7934c71b74d0b6542411258/users/68f87e1a83a34a90a750867f4b070840/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:24.358596 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1740e5-2812-478b-b88a-dea4314ffb5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b8f5d7ef7934c71b74d0b6542411258, user_id=68f87e1a83a34a90a750867f4b070840, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:24.361569 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1740e5-2812-478b-b88a-dea4314ffb5b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:24.377730 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 914/1842] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:42:24 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:42:24.392886 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 929/1843] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:24 2022] PUT /identity/v3/projects/0b8f5d7ef7934c71b74d0b6542411258/users/68f87e1a83a34a90a750867f4b070840/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:24.393460 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db24f9-ee3e-4af9-abc1-65a3cb70ad35 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:24.403466 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e7db24f9-ee3e-4af9-abc1-65a3cb70ad35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:24.404112 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e7ed1-254a-460a-903b-a896991b1061 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:24.404473 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e7ed1-254a-460a-903b-a896991b1061 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:24.404819 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e7ed1-254a-460a-903b-a896991b1061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:24.445564 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-289e7ed1-254a-460a-903b-a896991b1061 None None] MFA Rules not processed for user `68f87e1a83a34a90a750867f4b070840`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:24.447785 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-289e7ed1-254a-460a-903b-a896991b1061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:24.484088 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db24f9-ee3e-4af9-abc1-65a3cb70ad35 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:24.562955 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf54e19-7728-49bc-8601-3ffae0cb064b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:24.563183 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 930/1844] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:42:24 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:24.563291 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf54e19-7728-49bc-8601-3ffae0cb064b service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:24.563291 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf54e19-7728-49bc-8601-3ffae0cb064b service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:24.563291 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf54e19-7728-49bc-8601-3ffae0cb064b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:24.564659 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289e7ed1-254a-460a-903b-a896991b1061 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:24.634054 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:24.658492 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289e7ed1-254a-460a-903b-a896991b1061 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:24.664615 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9605ec1-7874-42d8-ac0a-e9b67e426c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:24.668059 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9605ec1-7874-42d8-ac0a-e9b67e426c22 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:24.669381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9605ec1-7874-42d8-ac0a-e9b67e426c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:24.669790 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf54e19-7728-49bc-8601-3ffae0cb064b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:24.670645 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9605ec1-7874-42d8-ac0a-e9b67e426c22 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:24.671359 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf54e19-7728-49bc-8601-3ffae0cb064b service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:24.671989 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9605ec1-7874-42d8-ac0a-e9b67e426c22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:24.674474 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9605ec1-7874-42d8-ac0a-e9b67e426c22 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:24.686077 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 915/1845] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:24 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:42:24.719703 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 931/1846] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:42:24 2022] POST /identity/v3/projects => generated 381 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:42:24.721323 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf54e19-7728-49bc-8601-3ffae0cb064b service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:24.737468 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf54e19-7728-49bc-8601-3ffae0cb064b service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:24.756245 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:24.785440 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9605ec1-7874-42d8-ac0a-e9b67e426c22 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:24.831622 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46140f74-734d-484f-8dd7-458e28ef2402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:24.832505 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46140f74-734d-484f-8dd7-458e28ef2402 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:24.832968 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46140f74-734d-484f-8dd7-458e28ef2402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:24.833587 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46140f74-734d-484f-8dd7-458e28ef2402 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:24.834459 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46140f74-734d-484f-8dd7-458e28ef2402 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:24.836927 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46140f74-734d-484f-8dd7-458e28ef2402 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:24.879432 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 916/1847] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:42:24 2022] POST /identity/v3/users => generated 297 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:24.881138 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46140f74-734d-484f-8dd7-458e28ef2402 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:24.894113 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46140f74-734d-484f-8dd7-458e28ef2402 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:24.900610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4aae1f-9da7-4368-9ee2-9f97a1f42845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:24.900785 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9605ec1-7874-42d8-ac0a-e9b67e426c22 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:24.902020 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4aae1f-9da7-4368-9ee2-9f97a1f42845 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:24.902802 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4aae1f-9da7-4368-9ee2-9f97a1f42845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:24.903543 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4aae1f-9da7-4368-9ee2-9f97a1f42845 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:24.905401 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4aae1f-9da7-4368-9ee2-9f97a1f42845 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:24.906378 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74df8e6-c3aa-4373-bb17-6e54eca107d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:24.907236 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74df8e6-c3aa-4373-bb17-6e54eca107d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:24.907521 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74df8e6-c3aa-4373-bb17-6e54eca107d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:24.907792 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74df8e6-c3aa-4373-bb17-6e54eca107d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:24.914249 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4aae1f-9da7-4368-9ee2-9f97a1f42845 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:24.918830 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e74df8e6-c3aa-4373-bb17-6e54eca107d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:24.933629 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 917/1848] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:24 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:24.935292 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4aae1f-9da7-4368-9ee2-9f97a1f42845 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:24.951032 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4aae1f-9da7-4368-9ee2-9f97a1f42845 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:24.958632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121d42a8-23f5-41bc-8318-b686185b0125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:24.959855 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121d42a8-23f5-41bc-8318-b686185b0125 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:24.960137 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121d42a8-23f5-41bc-8318-b686185b0125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:24.960964 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121d42a8-23f5-41bc-8318-b686185b0125 admin admin] PATH_INFO: `/v3/projects/884e5dd5b86c4776b87d5d27cecb5964/users/47c5a78fb93742f185868023d8fe86a5/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:24.987441 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121d42a8-23f5-41bc-8318-b686185b0125 admin admin] RBAC: Authorizing `identity:create_grant(project_id=884e5dd5b86c4776b87d5d27cecb5964, user_id=47c5a78fb93742f185868023d8fe86a5, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:24.992014 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121d42a8-23f5-41bc-8318-b686185b0125 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:25.034478 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 918/1849] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:24 2022] PUT /identity/v3/projects/884e5dd5b86c4776b87d5d27cecb5964/users/47c5a78fb93742f185868023d8fe86a5/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:25.040262 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121d42a8-23f5-41bc-8318-b686185b0125 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:25.121725 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74df8e6-c3aa-4373-bb17-6e54eca107d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:25.127107 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74df8e6-c3aa-4373-bb17-6e54eca107d2 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:25.158280 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 932/1850] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:42:25.196782 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121d42a8-23f5-41bc-8318-b686185b0125 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:25.220755 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588bb8b3-b9e7-46d6-810b-7d9d64e6d5e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:25.221137 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588bb8b3-b9e7-46d6-810b-7d9d64e6d5e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:25.223575 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588bb8b3-b9e7-46d6-810b-7d9d64e6d5e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:25.224054 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588bb8b3-b9e7-46d6-810b-7d9d64e6d5e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:25.225891 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588bb8b3-b9e7-46d6-810b-7d9d64e6d5e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:25.236089 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588bb8b3-b9e7-46d6-810b-7d9d64e6d5e5 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:25.262277 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 919/1851] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:25 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:25.267727 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74df8e6-c3aa-4373-bb17-6e54eca107d2 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:25.287938 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74df8e6-c3aa-4373-bb17-6e54eca107d2 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:25.307438 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf00f38c-8c1c-4c80-9f9a-9b7be0440da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:25.308161 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf00f38c-8c1c-4c80-9f9a-9b7be0440da9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:25.308595 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf00f38c-8c1c-4c80-9f9a-9b7be0440da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:25.308770 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf00f38c-8c1c-4c80-9f9a-9b7be0440da9 admin admin] PATH_INFO: `/v3/projects/884e5dd5b86c4776b87d5d27cecb5964/users/47c5a78fb93742f185868023d8fe86a5/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:25.311549 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf00f38c-8c1c-4c80-9f9a-9b7be0440da9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=884e5dd5b86c4776b87d5d27cecb5964, user_id=47c5a78fb93742f185868023d8fe86a5, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:25.314584 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf00f38c-8c1c-4c80-9f9a-9b7be0440da9 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:25.341995 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 933/1852] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:25 2022] PUT /identity/v3/projects/884e5dd5b86c4776b87d5d27cecb5964/users/47c5a78fb93742f185868023d8fe86a5/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:42:25.360066 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576db9d-9c70-4976-8a80-380daf9fc1ac None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:25.360533 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576db9d-9c70-4976-8a80-380daf9fc1ac None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:25.360953 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576db9d-9c70-4976-8a80-380daf9fc1ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:25.414105 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-5576db9d-9c70-4976-8a80-380daf9fc1ac None None] MFA Rules not processed for user `47c5a78fb93742f185868023d8fe86a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:25.415960 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-5576db9d-9c70-4976-8a80-380daf9fc1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:25.493937 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:25.514199 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 920/1853] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:42:25 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:25.527552 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf00f38c-8c1c-4c80-9f9a-9b7be0440da9 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:25.586879 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf00f38c-8c1c-4c80-9f9a-9b7be0440da9 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:25.591240 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f181a-bce4-42f2-905f-6065c6753e2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:25.591879 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f181a-bce4-42f2-905f-6065c6753e2b service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:25.592159 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f181a-bce4-42f2-905f-6065c6753e2b service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:25.592431 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f181a-bce4-42f2-905f-6065c6753e2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:25.601680 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3f181a-bce4-42f2-905f-6065c6753e2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:25.603148 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3f181a-bce4-42f2-905f-6065c6753e2b service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:25.613023 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 934/1854] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:25 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:42:26.040042 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5576db9d-9c70-4976-8a80-380daf9fc1ac None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:26.053717 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5576db9d-9c70-4976-8a80-380daf9fc1ac None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:26.060934 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee89eca-968c-40e0-b1ba-3f2b6d0af3d6 tempest-VolumeQuotaClassesTest-1471307078 tempest-VolumeQuotaClassesTest-1471307078-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47c5a78fb93742f185868023d8fe86a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '884e5dd5b86c4776b87d5d27cecb5964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:26.061597 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee89eca-968c-40e0-b1ba-3f2b6d0af3d6 tempest-VolumeQuotaClassesTest-1471307078 tempest-VolumeQuotaClassesTest-1471307078-project-admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:26.063038 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee89eca-968c-40e0-b1ba-3f2b6d0af3d6 tempest-VolumeQuotaClassesTest-1471307078 tempest-VolumeQuotaClassesTest-1471307078-project-admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:26.063328 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee89eca-968c-40e0-b1ba-3f2b6d0af3d6 tempest-VolumeQuotaClassesTest-1471307078 tempest-VolumeQuotaClassesTest-1471307078-project-admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:26.064193 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee89eca-968c-40e0-b1ba-3f2b6d0af3d6 tempest-VolumeQuotaClassesTest-1471307078 tempest-VolumeQuotaClassesTest-1471307078-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:42:26.064504 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee89eca-968c-40e0-b1ba-3f2b6d0af3d6 tempest-VolumeQuotaClassesTest-1471307078 tempest-VolumeQuotaClassesTest-1471307078-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:26.066161 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee89eca-968c-40e0-b1ba-3f2b6d0af3d6 tempest-VolumeQuotaClassesTest-1471307078 tempest-VolumeQuotaClassesTest-1471307078-project-admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:26.091264 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f181a-bce4-42f2-905f-6065c6753e2b service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:26.092929 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 921/1855] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:42:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:42:26.105585 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f181a-bce4-42f2-905f-6065c6753e2b service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:26.112327 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b284b-1167-48a9-b08b-660416abddc1 tempest-VolumeQuotaClassesTest-1471307078 tempest-VolumeQuotaClassesTest-1471307078-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47c5a78fb93742f185868023d8fe86a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '884e5dd5b86c4776b87d5d27cecb5964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:26.113113 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b284b-1167-48a9-b08b-660416abddc1 tempest-VolumeQuotaClassesTest-1471307078 tempest-VolumeQuotaClassesTest-1471307078-project-admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:26.113402 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b284b-1167-48a9-b08b-660416abddc1 tempest-VolumeQuotaClassesTest-1471307078 tempest-VolumeQuotaClassesTest-1471307078-project-admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:26.113676 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b284b-1167-48a9-b08b-660416abddc1 tempest-VolumeQuotaClassesTest-1471307078 tempest-VolumeQuotaClassesTest-1471307078-project-admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:26.114927 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44b284b-1167-48a9-b08b-660416abddc1 tempest-VolumeQuotaClassesTest-1471307078 tempest-VolumeQuotaClassesTest-1471307078-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:26.117030 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44b284b-1167-48a9-b08b-660416abddc1 tempest-VolumeQuotaClassesTest-1471307078 tempest-VolumeQuotaClassesTest-1471307078-project-admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:26.148845 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:26.183900 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee89eca-968c-40e0-b1ba-3f2b6d0af3d6 tempest-VolumeQuotaClassesTest-1471307078 tempest-VolumeQuotaClassesTest-1471307078-project-admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:26.190252 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 935/1856] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:42:26 2022] POST /identity/v3/projects => generated 352 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:42:26.196379 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee89eca-968c-40e0-b1ba-3f2b6d0af3d6 tempest-VolumeQuotaClassesTest-1471307078 tempest-VolumeQuotaClassesTest-1471307078-project-admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:26.255557 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f53b5-428f-4edf-8cac-4ec78b51d9e0 tempest-VolumeQuotaClassesTest-1471307078 tempest-VolumeQuotaClassesTest-1471307078-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47c5a78fb93742f185868023d8fe86a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '884e5dd5b86c4776b87d5d27cecb5964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:26.256759 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f53b5-428f-4edf-8cac-4ec78b51d9e0 tempest-VolumeQuotaClassesTest-1471307078 tempest-VolumeQuotaClassesTest-1471307078-project-admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:26.257046 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f53b5-428f-4edf-8cac-4ec78b51d9e0 tempest-VolumeQuotaClassesTest-1471307078 tempest-VolumeQuotaClassesTest-1471307078-project-admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:26.257317 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f53b5-428f-4edf-8cac-4ec78b51d9e0 tempest-VolumeQuotaClassesTest-1471307078 tempest-VolumeQuotaClassesTest-1471307078-project-admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:26.257996 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9f53b5-428f-4edf-8cac-4ec78b51d9e0 tempest-VolumeQuotaClassesTest-1471307078 tempest-VolumeQuotaClassesTest-1471307078-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:42:26.258309 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9f53b5-428f-4edf-8cac-4ec78b51d9e0 tempest-VolumeQuotaClassesTest-1471307078 tempest-VolumeQuotaClassesTest-1471307078-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:26.260280 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9f53b5-428f-4edf-8cac-4ec78b51d9e0 tempest-VolumeQuotaClassesTest-1471307078 tempest-VolumeQuotaClassesTest-1471307078-project-admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:26.274081 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 922/1857] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:42:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:42:26.321990 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b284b-1167-48a9-b08b-660416abddc1 tempest-VolumeQuotaClassesTest-1471307078 tempest-VolumeQuotaClassesTest-1471307078-project-admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:26.334449 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b284b-1167-48a9-b08b-660416abddc1 tempest-VolumeQuotaClassesTest-1471307078 tempest-VolumeQuotaClassesTest-1471307078-project-admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:26.340264 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c789a2-b84f-4316-8fc1-67626f7a8353 tempest-VolumeQuotaClassesTest-1471307078 tempest-VolumeQuotaClassesTest-1471307078-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47c5a78fb93742f185868023d8fe86a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '884e5dd5b86c4776b87d5d27cecb5964', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:26.340826 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c789a2-b84f-4316-8fc1-67626f7a8353 tempest-VolumeQuotaClassesTest-1471307078 tempest-VolumeQuotaClassesTest-1471307078-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:26.341102 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c789a2-b84f-4316-8fc1-67626f7a8353 tempest-VolumeQuotaClassesTest-1471307078 tempest-VolumeQuotaClassesTest-1471307078-project-admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:26.341372 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c789a2-b84f-4316-8fc1-67626f7a8353 tempest-VolumeQuotaClassesTest-1471307078 tempest-VolumeQuotaClassesTest-1471307078-project-admin] PATH_INFO: `/v3/projects/a1abf300b9cf4042929267c9f47ad186` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:26.342511 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c789a2-b84f-4316-8fc1-67626f7a8353 tempest-VolumeQuotaClassesTest-1471307078 tempest-VolumeQuotaClassesTest-1471307078-project-admin] RBAC: Authorizing `identity:delete_project(project_id=a1abf300b9cf4042929267c9f47ad186)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:26.344699 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c789a2-b84f-4316-8fc1-67626f7a8353 tempest-VolumeQuotaClassesTest-1471307078 tempest-VolumeQuotaClassesTest-1471307078-project-admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:26.399618 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-59c789a2-b84f-4316-8fc1-67626f7a8353 tempest-VolumeQuotaClassesTest-1471307078 tempest-VolumeQuotaClassesTest-1471307078-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1abf300b9cf4042929267c9f47ad186', 'request_id': 'req-59c789a2-b84f-4316-8fc1-67626f7a8353'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:26.407546 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-59c789a2-b84f-4316-8fc1-67626f7a8353 tempest-VolumeQuotaClassesTest-1471307078 tempest-VolumeQuotaClassesTest-1471307078-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1abf300b9cf4042929267c9f47ad186', 'request_id': 'req-59c789a2-b84f-4316-8fc1-67626f7a8353'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:26.416549 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-59c789a2-b84f-4316-8fc1-67626f7a8353 tempest-VolumeQuotaClassesTest-1471307078 tempest-VolumeQuotaClassesTest-1471307078-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1abf300b9cf4042929267c9f47ad186', 'request_id': 'req-59c789a2-b84f-4316-8fc1-67626f7a8353'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:26.427942 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-59c789a2-b84f-4316-8fc1-67626f7a8353 tempest-VolumeQuotaClassesTest-1471307078 tempest-VolumeQuotaClassesTest-1471307078-project-admin] The token cache is being invalidate because project a1abf300b9cf4042929267c9f47ad186 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:42:26.428357 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-59c789a2-b84f-4316-8fc1-67626f7a8353 tempest-VolumeQuotaClassesTest-1471307078 tempest-VolumeQuotaClassesTest-1471307078-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:26.429026 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-59c789a2-b84f-4316-8fc1-67626f7a8353 tempest-VolumeQuotaClassesTest-1471307078 tempest-VolumeQuotaClassesTest-1471307078-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:26.430832 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 936/1858] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:42:26 2022] DELETE /identity/v3/projects/a1abf300b9cf4042929267c9f47ad186 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:26.702283 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f53b5-428f-4edf-8cac-4ec78b51d9e0 tempest-VolumeQuotaClassesTest-1471307078 tempest-VolumeQuotaClassesTest-1471307078-project-admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:26.704050 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-6e9f53b5-428f-4edf-8cac-4ec78b51d9e0 tempest-VolumeQuotaClassesTest-1471307078 tempest-VolumeQuotaClassesTest-1471307078-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:26.797578 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f53b5-428f-4edf-8cac-4ec78b51d9e0 tempest-VolumeQuotaClassesTest-1471307078 tempest-VolumeQuotaClassesTest-1471307078-project-admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:26.892497 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af9d8ae-6e7b-495d-b4f7-bcbbcab48b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:26.893119 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af9d8ae-6e7b-495d-b4f7-bcbbcab48b35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:26.893405 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af9d8ae-6e7b-495d-b4f7-bcbbcab48b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:26.893685 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af9d8ae-6e7b-495d-b4f7-bcbbcab48b35 admin admin] PATH_INFO: `/v3/users/68f87e1a83a34a90a750867f4b070840` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:26.894923 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af9d8ae-6e7b-495d-b4f7-bcbbcab48b35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68f87e1a83a34a90a750867f4b070840)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:26.899315 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af9d8ae-6e7b-495d-b4f7-bcbbcab48b35 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:26.954349 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:27.005321 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7af9d8ae-6e7b-495d-b4f7-bcbbcab48b35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68f87e1a83a34a90a750867f4b070840'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:27.014297 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7af9d8ae-6e7b-495d-b4f7-bcbbcab48b35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68f87e1a83a34a90a750867f4b070840'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:27.015174 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7af9d8ae-6e7b-495d-b4f7-bcbbcab48b35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68f87e1a83a34a90a750867f4b070840'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:27.035484 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7af9d8ae-6e7b-495d-b4f7-bcbbcab48b35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68f87e1a83a34a90a750867f4b070840'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:27.036254 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7af9d8ae-6e7b-495d-b4f7-bcbbcab48b35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68f87e1a83a34a90a750867f4b070840'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:27.060097 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 923/1859] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:42:26 2022] DELETE /identity/v3/users/68f87e1a83a34a90a750867f4b070840 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:27.064325 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa0cfb7-64d4-4970-a366-ba566109c52b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:27.064614 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa0cfb7-64d4-4970-a366-ba566109c52b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:27.064891 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa0cfb7-64d4-4970-a366-ba566109c52b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:27.084686 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-daa0cfb7-64d4-4970-a366-ba566109c52b None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:27.085972 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-daa0cfb7-64d4-4970-a366-ba566109c52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:27.157066 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 937/1860] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:42:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:42:27.172009 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af9d8ae-6e7b-495d-b4f7-bcbbcab48b35 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:27.174061 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7af9d8ae-6e7b-495d-b4f7-bcbbcab48b35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:27.240385 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af9d8ae-6e7b-495d-b4f7-bcbbcab48b35 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:27.266024 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1791621-a9a2-4f8e-b54e-e4f8753654cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:27.266024 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1791621-a9a2-4f8e-b54e-e4f8753654cb service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:27.266024 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1791621-a9a2-4f8e-b54e-e4f8753654cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:27.266024 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1791621-a9a2-4f8e-b54e-e4f8753654cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:27.274653 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1791621-a9a2-4f8e-b54e-e4f8753654cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:27.279324 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1791621-a9a2-4f8e-b54e-e4f8753654cb service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:27.301076 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 924/1861] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:42:28.182292 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa0cfb7-64d4-4970-a366-ba566109c52b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:28.186249 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-daa0cfb7-64d4-4970-a366-ba566109c52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:28.214480 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa0cfb7-64d4-4970-a366-ba566109c52b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:28.222855 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eeb8b6-78c6-4ee1-af05-9b4c9a6043ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:28.223507 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eeb8b6-78c6-4ee1-af05-9b4c9a6043ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:28.223964 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eeb8b6-78c6-4ee1-af05-9b4c9a6043ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:28.224386 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eeb8b6-78c6-4ee1-af05-9b4c9a6043ed admin admin] PATH_INFO: `/v3/projects/0b8f5d7ef7934c71b74d0b6542411258` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:28.226021 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eeb8b6-78c6-4ee1-af05-9b4c9a6043ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b8f5d7ef7934c71b74d0b6542411258)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:28.238570 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eeb8b6-78c6-4ee1-af05-9b4c9a6043ed admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:28.313688 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-35eeb8b6-78c6-4ee1-af05-9b4c9a6043ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b8f5d7ef7934c71b74d0b6542411258', 'request_id': 'req-35eeb8b6-78c6-4ee1-af05-9b4c9a6043ed'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:28.333259 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-35eeb8b6-78c6-4ee1-af05-9b4c9a6043ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b8f5d7ef7934c71b74d0b6542411258', 'request_id': 'req-35eeb8b6-78c6-4ee1-af05-9b4c9a6043ed'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:28.351096 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-35eeb8b6-78c6-4ee1-af05-9b4c9a6043ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b8f5d7ef7934c71b74d0b6542411258', 'request_id': 'req-35eeb8b6-78c6-4ee1-af05-9b4c9a6043ed'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:28.369671 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-35eeb8b6-78c6-4ee1-af05-9b4c9a6043ed admin admin] The token cache is being invalidate because project 0b8f5d7ef7934c71b74d0b6542411258 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:42:28.371030 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-35eeb8b6-78c6-4ee1-af05-9b4c9a6043ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:28.372143 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-35eeb8b6-78c6-4ee1-af05-9b4c9a6043ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:28.378725 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1791621-a9a2-4f8e-b54e-e4f8753654cb service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:28.379263 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 938/1862] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:42:28 2022] DELETE /identity/v3/projects/0b8f5d7ef7934c71b74d0b6542411258 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:28.393025 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1791621-a9a2-4f8e-b54e-e4f8753654cb service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:28.475267 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38363e6c-4ced-4e59-b202-6aa2ac71fac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:28.475970 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38363e6c-4ced-4e59-b202-6aa2ac71fac6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:28.476312 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38363e6c-4ced-4e59-b202-6aa2ac71fac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:28.476645 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38363e6c-4ced-4e59-b202-6aa2ac71fac6 admin admin] PATH_INFO: `/v3/users/47c5a78fb93742f185868023d8fe86a5` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:28.477954 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38363e6c-4ced-4e59-b202-6aa2ac71fac6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47c5a78fb93742f185868023d8fe86a5)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:28.480590 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38363e6c-4ced-4e59-b202-6aa2ac71fac6 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:28.548221 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:28.550397 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-38363e6c-4ced-4e59-b202-6aa2ac71fac6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47c5a78fb93742f185868023d8fe86a5'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:28.562617 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-38363e6c-4ced-4e59-b202-6aa2ac71fac6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47c5a78fb93742f185868023d8fe86a5'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:28.563573 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-38363e6c-4ced-4e59-b202-6aa2ac71fac6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47c5a78fb93742f185868023d8fe86a5'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:28.576792 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-38363e6c-4ced-4e59-b202-6aa2ac71fac6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47c5a78fb93742f185868023d8fe86a5'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:28.577478 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-38363e6c-4ced-4e59-b202-6aa2ac71fac6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47c5a78fb93742f185868023d8fe86a5'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:28.607250 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 925/1863] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:42:28 2022] DELETE /identity/v3/users/47c5a78fb93742f185868023d8fe86a5 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:42:29.042610 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eeb8b6-78c6-4ee1-af05-9b4c9a6043ed admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:29.045289 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-35eeb8b6-78c6-4ee1-af05-9b4c9a6043ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:29.060011 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eeb8b6-78c6-4ee1-af05-9b4c9a6043ed admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:29.064681 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:29.132202 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc3d3b-b4e5-4528-8381-82e9553f03d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:29.132981 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc3d3b-b4e5-4528-8381-82e9553f03d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:29.133731 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc3d3b-b4e5-4528-8381-82e9553f03d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:29.133731 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc3d3b-b4e5-4528-8381-82e9553f03d7 admin admin] PATH_INFO: `/v3/projects/884e5dd5b86c4776b87d5d27cecb5964` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:29.135019 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edc3d3b-b4e5-4528-8381-82e9553f03d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=884e5dd5b86c4776b87d5d27cecb5964)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:29.137546 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edc3d3b-b4e5-4528-8381-82e9553f03d7 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:29.199542 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1edc3d3b-b4e5-4528-8381-82e9553f03d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '884e5dd5b86c4776b87d5d27cecb5964', 'request_id': 'req-1edc3d3b-b4e5-4528-8381-82e9553f03d7'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:29.211596 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1edc3d3b-b4e5-4528-8381-82e9553f03d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '884e5dd5b86c4776b87d5d27cecb5964', 'request_id': 'req-1edc3d3b-b4e5-4528-8381-82e9553f03d7'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:29.221772 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1edc3d3b-b4e5-4528-8381-82e9553f03d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '884e5dd5b86c4776b87d5d27cecb5964', 'request_id': 'req-1edc3d3b-b4e5-4528-8381-82e9553f03d7'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:29.235500 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1edc3d3b-b4e5-4528-8381-82e9553f03d7 admin admin] The token cache is being invalidate because project 884e5dd5b86c4776b87d5d27cecb5964 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:42:29.235912 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1edc3d3b-b4e5-4528-8381-82e9553f03d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:29.236632 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1edc3d3b-b4e5-4528-8381-82e9553f03d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:29.241950 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 939/1864] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:42:29 2022] DELETE /identity/v3/projects/884e5dd5b86c4776b87d5d27cecb5964 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:42:29.243581 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2ec2c-547b-425a-acb6-85d9e174d712 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:29.243864 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2ec2c-547b-425a-acb6-85d9e174d712 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:29.244204 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2ec2c-547b-425a-acb6-85d9e174d712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:29.266256 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-bbd2ec2c-547b-425a-acb6-85d9e174d712 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:29.267553 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-bbd2ec2c-547b-425a-acb6-85d9e174d712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:29.330759 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 926/1865] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:42:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:29.339338 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af3090a-e4b8-4eea-b400-f5e9b207f94f None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:29.339629 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af3090a-e4b8-4eea-b400-f5e9b207f94f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:29.339907 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af3090a-e4b8-4eea-b400-f5e9b207f94f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:29.359586 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-5af3090a-e4b8-4eea-b400-f5e9b207f94f None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:29.360792 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5af3090a-e4b8-4eea-b400-f5e9b207f94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:29.368635 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 940/1866] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:42:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:42:29.372867 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd2ec2c-547b-425a-acb6-85d9e174d712 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:29.382806 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd2ec2c-547b-425a-acb6-85d9e174d712 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:29.387946 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68f8024-6715-4656-840c-09a8ff78ecd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:29.388730 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f8024-6715-4656-840c-09a8ff78ecd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:29.389009 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f8024-6715-4656-840c-09a8ff78ecd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:29.389275 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f8024-6715-4656-840c-09a8ff78ecd4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:29.390007 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68f8024-6715-4656-840c-09a8ff78ecd4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:42:29.390305 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68f8024-6715-4656-840c-09a8ff78ecd4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:29.391878 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68f8024-6715-4656-840c-09a8ff78ecd4 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:29.407522 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 927/1867] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:42:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:42:29.408681 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af3090a-e4b8-4eea-b400-f5e9b207f94f None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:29.418031 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af3090a-e4b8-4eea-b400-f5e9b207f94f None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:29.422706 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aedf652-b9cb-46a6-8af1-08c634ddf3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:29.423302 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aedf652-b9cb-46a6-8af1-08c634ddf3f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:29.423575 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aedf652-b9cb-46a6-8af1-08c634ddf3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:29.423844 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aedf652-b9cb-46a6-8af1-08c634ddf3f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:29.424455 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aedf652-b9cb-46a6-8af1-08c634ddf3f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:29.426380 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aedf652-b9cb-46a6-8af1-08c634ddf3f6 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:29.459078 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 941/1868] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:42:29 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:29.460539 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68f8024-6715-4656-840c-09a8ff78ecd4 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:29.475430 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68f8024-6715-4656-840c-09a8ff78ecd4 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:29.536500 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0395a816-efe9-4b1f-aee5-36c7d27ccbf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:29.537112 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395a816-efe9-4b1f-aee5-36c7d27ccbf0 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:29.537392 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395a816-efe9-4b1f-aee5-36c7d27ccbf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:29.537663 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0395a816-efe9-4b1f-aee5-36c7d27ccbf0 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:29.538333 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0395a816-efe9-4b1f-aee5-36c7d27ccbf0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:29.540281 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0395a816-efe9-4b1f-aee5-36c7d27ccbf0 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:29.566987 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 928/1869] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:42:29 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:29.568276 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aedf652-b9cb-46a6-8af1-08c634ddf3f6 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:29.579718 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aedf652-b9cb-46a6-8af1-08c634ddf3f6 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:29.585629 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4930e6d2-9055-4cfb-8675-52178f230690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:29.586268 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930e6d2-9055-4cfb-8675-52178f230690 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:29.586545 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930e6d2-9055-4cfb-8675-52178f230690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:29.586838 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930e6d2-9055-4cfb-8675-52178f230690 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:29.587468 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4930e6d2-9055-4cfb-8675-52178f230690 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:29.589145 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4930e6d2-9055-4cfb-8675-52178f230690 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:29.599711 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 942/1870] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:29.603895 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0395a816-efe9-4b1f-aee5-36c7d27ccbf0 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:29.614578 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0395a816-efe9-4b1f-aee5-36c7d27ccbf0 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:29.619335 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce8745a-d3df-4609-8022-beaae8098f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:29.620517 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8745a-d3df-4609-8022-beaae8098f86 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:29.620795 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8745a-d3df-4609-8022-beaae8098f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:29.621064 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8745a-d3df-4609-8022-beaae8098f86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:29.621736 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce8745a-d3df-4609-8022-beaae8098f86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:29.623660 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce8745a-d3df-4609-8022-beaae8098f86 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:29.636801 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 929/1871] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:29.640348 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4930e6d2-9055-4cfb-8675-52178f230690 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:29.651161 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4930e6d2-9055-4cfb-8675-52178f230690 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:29.656066 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c443855-b087-4a0b-ac36-68861d1308f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:29.656740 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c443855-b087-4a0b-ac36-68861d1308f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:29.657018 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c443855-b087-4a0b-ac36-68861d1308f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:29.657287 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c443855-b087-4a0b-ac36-68861d1308f1 admin admin] PATH_INFO: `/v3/projects/170cac2f0a4a429da386ddaa0642df8c/users/1c5eabeda8234861a28d38cb3839bec6/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:29.674294 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c443855-b087-4a0b-ac36-68861d1308f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=170cac2f0a4a429da386ddaa0642df8c, user_id=1c5eabeda8234861a28d38cb3839bec6, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:29.677447 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c443855-b087-4a0b-ac36-68861d1308f1 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:29.729561 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 943/1872] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:29 2022] PUT /identity/v3/projects/170cac2f0a4a429da386ddaa0642df8c/users/1c5eabeda8234861a28d38cb3839bec6/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:29.743173 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e323069-1dc4-43ba-83f8-53b78fc1f716 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:29.743532 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e323069-1dc4-43ba-83f8-53b78fc1f716 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:29.744137 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e323069-1dc4-43ba-83f8-53b78fc1f716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:29.779755 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-8e323069-1dc4-43ba-83f8-53b78fc1f716 None None] MFA Rules not processed for user `1c5eabeda8234861a28d38cb3839bec6`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:29.781082 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-8e323069-1dc4-43ba-83f8-53b78fc1f716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:29.803793 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:29.863427 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 930/1873] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:42:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:42:29.864583 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c443855-b087-4a0b-ac36-68861d1308f1 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:29.910216 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:29.960729 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c443855-b087-4a0b-ac36-68861d1308f1 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:29.982739 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07853424-6db7-410a-8749-5976281a1ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:29.983651 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07853424-6db7-410a-8749-5976281a1ae5 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:29.984075 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07853424-6db7-410a-8749-5976281a1ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:29.984489 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07853424-6db7-410a-8749-5976281a1ae5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:29.986400 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07853424-6db7-410a-8749-5976281a1ae5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:29.989750 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07853424-6db7-410a-8749-5976281a1ae5 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:30.042516 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 944/1874] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:42:29 2022] POST /identity/v3/projects => generated 385 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:30.047409 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e323069-1dc4-43ba-83f8-53b78fc1f716 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:30.063527 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e323069-1dc4-43ba-83f8-53b78fc1f716 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:30.136732 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f24da8b-78e5-4bb5-9d31-3753bb4e0113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:30.137359 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f24da8b-78e5-4bb5-9d31-3753bb4e0113 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:30.137639 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f24da8b-78e5-4bb5-9d31-3753bb4e0113 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:30.138130 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f24da8b-78e5-4bb5-9d31-3753bb4e0113 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:30.138580 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f24da8b-78e5-4bb5-9d31-3753bb4e0113 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:30.141032 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f24da8b-78e5-4bb5-9d31-3753bb4e0113 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:30.165984 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 931/1875] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:42:30 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:30.170310 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07853424-6db7-410a-8749-5976281a1ae5 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:30.182122 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07853424-6db7-410a-8749-5976281a1ae5 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:30.188224 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d021472e-7d82-4f34-a365-6969a354ad5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:30.188895 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021472e-7d82-4f34-a365-6969a354ad5d admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:30.189320 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021472e-7d82-4f34-a365-6969a354ad5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:30.189627 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021472e-7d82-4f34-a365-6969a354ad5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:30.190392 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d021472e-7d82-4f34-a365-6969a354ad5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:30.192289 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d021472e-7d82-4f34-a365-6969a354ad5d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:30.208174 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 945/1876] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:42:30.209390 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f24da8b-78e5-4bb5-9d31-3753bb4e0113 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:30.220484 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f24da8b-78e5-4bb5-9d31-3753bb4e0113 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:30.229779 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e44bf-6eb1-4aa8-a856-7827716c7c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:30.230452 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e44bf-6eb1-4aa8-a856-7827716c7c8e admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:30.230724 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e44bf-6eb1-4aa8-a856-7827716c7c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:30.230992 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e44bf-6eb1-4aa8-a856-7827716c7c8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:30.231633 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425e44bf-6eb1-4aa8-a856-7827716c7c8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:30.233211 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425e44bf-6eb1-4aa8-a856-7827716c7c8e admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:30.245482 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 932/1877] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:30.248449 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d021472e-7d82-4f34-a365-6969a354ad5d admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:30.259058 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d021472e-7d82-4f34-a365-6969a354ad5d admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:30.263933 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e6823b-3ff4-4b86-be3b-f0d081b46e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:30.264601 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6823b-3ff4-4b86-be3b-f0d081b46e44 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:30.264877 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6823b-3ff4-4b86-be3b-f0d081b46e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:30.265148 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6823b-3ff4-4b86-be3b-f0d081b46e44 admin admin] PATH_INFO: `/v3/projects/19605a281cc340069680d5a73ec08e7a/users/15405b3ed20b4f749d30755f00665e73/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:30.286037 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e6823b-3ff4-4b86-be3b-f0d081b46e44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19605a281cc340069680d5a73ec08e7a, user_id=15405b3ed20b4f749d30755f00665e73, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:30.289220 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e6823b-3ff4-4b86-be3b-f0d081b46e44 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:30.309712 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 946/1878] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:30 2022] PUT /identity/v3/projects/19605a281cc340069680d5a73ec08e7a/users/15405b3ed20b4f749d30755f00665e73/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:30.320020 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d30e53-d845-440d-8e84-13a89992e816 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:30.320302 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d30e53-d845-440d-8e84-13a89992e816 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:30.320577 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d30e53-d845-440d-8e84-13a89992e816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:30.352479 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-51d30e53-d845-440d-8e84-13a89992e816 None None] MFA Rules not processed for user `15405b3ed20b4f749d30755f00665e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:30.353714 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-51d30e53-d845-440d-8e84-13a89992e816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:30.436079 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 933/1879] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:42:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:30.440794 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e6823b-3ff4-4b86-be3b-f0d081b46e44 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:30.505591 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e6823b-3ff4-4b86-be3b-f0d081b46e44 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:30.510429 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c502c5-75e0-45ac-a7e4-c8291fbba9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:30.511039 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c502c5-75e0-45ac-a7e4-c8291fbba9c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:30.511319 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c502c5-75e0-45ac-a7e4-c8291fbba9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:30.511591 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c502c5-75e0-45ac-a7e4-c8291fbba9c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:30.512247 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c502c5-75e0-45ac-a7e4-c8291fbba9c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:30.514293 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c502c5-75e0-45ac-a7e4-c8291fbba9c8 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:30.545662 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 947/1880] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:42:30 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:42:30.546868 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d30e53-d845-440d-8e84-13a89992e816 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:30.557751 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d30e53-d845-440d-8e84-13a89992e816 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:30.582273 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:30.615828 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecae255-a5cf-4341-a440-cd6962698284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:30.616361 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecae255-a5cf-4341-a440-cd6962698284 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:30.616635 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecae255-a5cf-4341-a440-cd6962698284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:30.616898 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecae255-a5cf-4341-a440-cd6962698284 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:30.617512 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecae255-a5cf-4341-a440-cd6962698284 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:30.619436 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecae255-a5cf-4341-a440-cd6962698284 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:30.640387 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 934/1881] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:42:30 2022] POST /identity/v3/users => generated 300 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:30.644422 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c502c5-75e0-45ac-a7e4-c8291fbba9c8 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:30.655332 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c502c5-75e0-45ac-a7e4-c8291fbba9c8 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:30.659960 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dac9eaf-60bf-473d-beaf-6fe09c64b335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:30.660555 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac9eaf-60bf-473d-beaf-6fe09c64b335 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:30.660834 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac9eaf-60bf-473d-beaf-6fe09c64b335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:30.661113 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac9eaf-60bf-473d-beaf-6fe09c64b335 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:30.661743 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dac9eaf-60bf-473d-beaf-6fe09c64b335 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:30.663397 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dac9eaf-60bf-473d-beaf-6fe09c64b335 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:30.673672 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 948/1882] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:30.677653 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecae255-a5cf-4341-a440-cd6962698284 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:30.687847 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecae255-a5cf-4341-a440-cd6962698284 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:30.692679 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5800617a-4c3c-4eff-b8e4-ca7c86c77580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:30.693369 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5800617a-4c3c-4eff-b8e4-ca7c86c77580 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:30.693673 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5800617a-4c3c-4eff-b8e4-ca7c86c77580 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:30.693986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5800617a-4c3c-4eff-b8e4-ca7c86c77580 admin admin] PATH_INFO: `/v3/projects/b972ae166c384e7fae3a4b7040c1a7a7/users/3348c0100afa4fc19406ec9690c89572/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:30.710052 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5800617a-4c3c-4eff-b8e4-ca7c86c77580 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b972ae166c384e7fae3a4b7040c1a7a7, user_id=3348c0100afa4fc19406ec9690c89572, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:30.712691 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5800617a-4c3c-4eff-b8e4-ca7c86c77580 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:30.725640 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 935/1883] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:30 2022] PUT /identity/v3/projects/b972ae166c384e7fae3a4b7040c1a7a7/users/3348c0100afa4fc19406ec9690c89572/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:30.729500 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dac9eaf-60bf-473d-beaf-6fe09c64b335 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:30.739603 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dac9eaf-60bf-473d-beaf-6fe09c64b335 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:30.789520 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:30.796578 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c878688-9c52-4e06-8183-8971a7c3bd94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:30.797324 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c878688-9c52-4e06-8183-8971a7c3bd94 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:30.797834 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c878688-9c52-4e06-8183-8971a7c3bd94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:30.798321 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c878688-9c52-4e06-8183-8971a7c3bd94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:30.799173 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c878688-9c52-4e06-8183-8971a7c3bd94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:30.800876 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c878688-9c52-4e06-8183-8971a7c3bd94 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:30.810658 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 949/1884] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:30 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:42:30.814849 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5800617a-4c3c-4eff-b8e4-ca7c86c77580 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:30.825462 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5800617a-4c3c-4eff-b8e4-ca7c86c77580 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:30.830018 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91621aae-dbcf-4c44-ab2c-f56d2d5ef209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:30.830664 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91621aae-dbcf-4c44-ab2c-f56d2d5ef209 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:30.830942 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91621aae-dbcf-4c44-ab2c-f56d2d5ef209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:30.831214 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91621aae-dbcf-4c44-ab2c-f56d2d5ef209 admin admin] PATH_INFO: `/v3/projects/b972ae166c384e7fae3a4b7040c1a7a7/users/3348c0100afa4fc19406ec9690c89572/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:30.833595 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91621aae-dbcf-4c44-ab2c-f56d2d5ef209 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b972ae166c384e7fae3a4b7040c1a7a7, user_id=3348c0100afa4fc19406ec9690c89572, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:30.836495 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91621aae-dbcf-4c44-ab2c-f56d2d5ef209 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:30.847389 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 936/1885] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:30 2022] PUT /identity/v3/projects/b972ae166c384e7fae3a4b7040c1a7a7/users/3348c0100afa4fc19406ec9690c89572/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:42:30.857280 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d4dd2-2ca4-473e-921f-c98a9e4390ff None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:30.857567 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d4dd2-2ca4-473e-921f-c98a9e4390ff None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:30.857843 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d4dd2-2ca4-473e-921f-c98a9e4390ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:30.884810 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-b72d4dd2-2ca4-473e-921f-c98a9e4390ff None None] MFA Rules not processed for user `3348c0100afa4fc19406ec9690c89572`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:30.886498 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b72d4dd2-2ca4-473e-921f-c98a9e4390ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:30.958204 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 950/1886] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:42:30 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:30.966547 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91621aae-dbcf-4c44-ab2c-f56d2d5ef209 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:30.968901 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-91621aae-dbcf-4c44-ab2c-f56d2d5ef209 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:31.031633 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91621aae-dbcf-4c44-ab2c-f56d2d5ef209 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:31.035311 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afc0be8-891e-43e9-b6bb-7d6dd4190b36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:31.035982 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc0be8-891e-43e9-b6bb-7d6dd4190b36 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:31.036317 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc0be8-891e-43e9-b6bb-7d6dd4190b36 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:31.036650 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc0be8-891e-43e9-b6bb-7d6dd4190b36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:31.046920 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afc0be8-891e-43e9-b6bb-7d6dd4190b36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:31.048542 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afc0be8-891e-43e9-b6bb-7d6dd4190b36 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:31.060210 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 937/1887] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:30 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:42:31.616135 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72d4dd2-2ca4-473e-921f-c98a9e4390ff None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:31.625804 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72d4dd2-2ca4-473e-921f-c98a9e4390ff None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:31.629344 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846fe213-2faf-4ea3-86c8-3bb92d850d2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:31.630044 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846fe213-2faf-4ea3-86c8-3bb92d850d2d service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:31.630342 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846fe213-2faf-4ea3-86c8-3bb92d850d2d service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:31.630628 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846fe213-2faf-4ea3-86c8-3bb92d850d2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:31.693612 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846fe213-2faf-4ea3-86c8-3bb92d850d2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:31.695113 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846fe213-2faf-4ea3-86c8-3bb92d850d2d service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:31.704921 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 951/1888] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:31 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:42:35.359786 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afc0be8-891e-43e9-b6bb-7d6dd4190b36 service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:35.362269 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4afc0be8-891e-43e9-b6bb-7d6dd4190b36 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:35.454074 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afc0be8-891e-43e9-b6bb-7d6dd4190b36 service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:35.459132 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1846d0-8fe3-475f-a4b9-f1c3cec1d795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:35.459958 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1846d0-8fe3-475f-a4b9-f1c3cec1d795 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:35.460233 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1846d0-8fe3-475f-a4b9-f1c3cec1d795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:35.460500 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1846d0-8fe3-475f-a4b9-f1c3cec1d795 admin admin] PATH_INFO: `/v3/users/207922b68d704a18bc4fe64de88775f0` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:35.461731 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1846d0-8fe3-475f-a4b9-f1c3cec1d795 admin admin] RBAC: Authorizing `identity:delete_user(user_id=207922b68d704a18bc4fe64de88775f0)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:35.464218 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1846d0-8fe3-475f-a4b9-f1c3cec1d795 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:35.494012 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:35.521644 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1e1846d0-8fe3-475f-a4b9-f1c3cec1d795 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '207922b68d704a18bc4fe64de88775f0'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:35.529686 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1e1846d0-8fe3-475f-a4b9-f1c3cec1d795 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '207922b68d704a18bc4fe64de88775f0'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:35.530416 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1e1846d0-8fe3-475f-a4b9-f1c3cec1d795 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '207922b68d704a18bc4fe64de88775f0'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:35.538024 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1e1846d0-8fe3-475f-a4b9-f1c3cec1d795 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '207922b68d704a18bc4fe64de88775f0'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:35.538716 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1e1846d0-8fe3-475f-a4b9-f1c3cec1d795 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '207922b68d704a18bc4fe64de88775f0'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:35.561811 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 938/1889] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:42:35 2022] DELETE /identity/v3/users/207922b68d704a18bc4fe64de88775f0 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:42:35.566285 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf31f9-8b97-464e-b9dd-a82e9d777bc8 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:35.566782 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf31f9-8b97-464e-b9dd-a82e9d777bc8 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:35.566999 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf31f9-8b97-464e-b9dd-a82e9d777bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:35.585905 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-8fbf31f9-8b97-464e-b9dd-a82e9d777bc8 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:35.587094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8fbf31f9-8b97-464e-b9dd-a82e9d777bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:35.656946 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 952/1890] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:42:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:42:35.667964 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1846d0-8fe3-475f-a4b9-f1c3cec1d795 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:35.669377 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-1e1846d0-8fe3-475f-a4b9-f1c3cec1d795 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:35.729742 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1846d0-8fe3-475f-a4b9-f1c3cec1d795 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:35.733458 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a7a48-b889-4db1-8da5-a79fb8a897f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:35.734098 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a7a48-b889-4db1-8da5-a79fb8a897f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:35.734370 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a7a48-b889-4db1-8da5-a79fb8a897f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:35.734636 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a7a48-b889-4db1-8da5-a79fb8a897f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:35.743675 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72a7a48-b889-4db1-8da5-a79fb8a897f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:35.745098 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72a7a48-b889-4db1-8da5-a79fb8a897f2 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:35.755770 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 939/1891] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:42:36.127913 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbf31f9-8b97-464e-b9dd-a82e9d777bc8 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:36.129810 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8fbf31f9-8b97-464e-b9dd-a82e9d777bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:36.152930 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:36.194102 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbf31f9-8b97-464e-b9dd-a82e9d777bc8 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:36.198367 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e44cc8-7d98-444e-9f8d-78836444df52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:36.199270 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e44cc8-7d98-444e-9f8d-78836444df52 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:36.199634 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e44cc8-7d98-444e-9f8d-78836444df52 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:36.200301 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e44cc8-7d98-444e-9f8d-78836444df52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:36.202522 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-40e44cc8-7d98-444e-9f8d-78836444df52 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:36.334240 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e44cc8-7d98-444e-9f8d-78836444df52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:36.340302 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e44cc8-7d98-444e-9f8d-78836444df52 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:36.361282 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 953/1892] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:36 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:42:36.384154 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a7a48-b889-4db1-8da5-a79fb8a897f2 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:36.386395 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c72a7a48-b889-4db1-8da5-a79fb8a897f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:36.404181 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a7a48-b889-4db1-8da5-a79fb8a897f2 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:36.409290 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316133ec-386a-4505-a489-22147ecab6c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:36.410181 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316133ec-386a-4505-a489-22147ecab6c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:36.410660 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316133ec-386a-4505-a489-22147ecab6c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:36.411070 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316133ec-386a-4505-a489-22147ecab6c1 admin admin] PATH_INFO: `/v3/projects/edddf0c9177b46cf9ca89f87543a4e13` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:36.412368 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316133ec-386a-4505-a489-22147ecab6c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edddf0c9177b46cf9ca89f87543a4e13)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:36.414757 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316133ec-386a-4505-a489-22147ecab6c1 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:36.474768 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-316133ec-386a-4505-a489-22147ecab6c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edddf0c9177b46cf9ca89f87543a4e13', 'request_id': 'req-316133ec-386a-4505-a489-22147ecab6c1'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:36.504238 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-316133ec-386a-4505-a489-22147ecab6c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edddf0c9177b46cf9ca89f87543a4e13', 'request_id': 'req-316133ec-386a-4505-a489-22147ecab6c1'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:36.515010 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-316133ec-386a-4505-a489-22147ecab6c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edddf0c9177b46cf9ca89f87543a4e13', 'request_id': 'req-316133ec-386a-4505-a489-22147ecab6c1'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:36.530426 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-316133ec-386a-4505-a489-22147ecab6c1 admin admin] The token cache is being invalidate because project edddf0c9177b46cf9ca89f87543a4e13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:42:36.530816 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-316133ec-386a-4505-a489-22147ecab6c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:36.532238 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-316133ec-386a-4505-a489-22147ecab6c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:36.536975 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 940/1893] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:42:36 2022] DELETE /identity/v3/projects/edddf0c9177b46cf9ca89f87543a4e13 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:36.538589 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e44cc8-7d98-444e-9f8d-78836444df52 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:36.551244 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e44cc8-7d98-444e-9f8d-78836444df52 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:36.681096 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4305e1c-0e76-40d0-a0b7-b5c9cd6344d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:36.683058 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4305e1c-0e76-40d0-a0b7-b5c9cd6344d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:36.683473 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4305e1c-0e76-40d0-a0b7-b5c9cd6344d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:36.683760 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4305e1c-0e76-40d0-a0b7-b5c9cd6344d9 admin admin] PATH_INFO: `/v3/users/ca5944ec970446068f9149e7b8e09443` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:36.685412 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4305e1c-0e76-40d0-a0b7-b5c9cd6344d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca5944ec970446068f9149e7b8e09443)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:36.687380 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4305e1c-0e76-40d0-a0b7-b5c9cd6344d9 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:36.750905 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c4305e1c-0e76-40d0-a0b7-b5c9cd6344d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca5944ec970446068f9149e7b8e09443'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:36.751872 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c4305e1c-0e76-40d0-a0b7-b5c9cd6344d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca5944ec970446068f9149e7b8e09443'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:36.771563 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c4305e1c-0e76-40d0-a0b7-b5c9cd6344d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca5944ec970446068f9149e7b8e09443'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:36.783780 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c4305e1c-0e76-40d0-a0b7-b5c9cd6344d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca5944ec970446068f9149e7b8e09443'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:36.789197 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:36.798327 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c4305e1c-0e76-40d0-a0b7-b5c9cd6344d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca5944ec970446068f9149e7b8e09443'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:36.807618 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 954/1894] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:42:36 2022] DELETE /identity/v3/users/ca5944ec970446068f9149e7b8e09443 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:37.099493 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316133ec-386a-4505-a489-22147ecab6c1 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:37.101240 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-316133ec-386a-4505-a489-22147ecab6c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:37.137644 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316133ec-386a-4505-a489-22147ecab6c1 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:37.173100 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:37.230416 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835f8b01-4556-40b8-890c-95ef7e28ca72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:37.231937 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f8b01-4556-40b8-890c-95ef7e28ca72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:37.232224 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f8b01-4556-40b8-890c-95ef7e28ca72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:37.232509 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f8b01-4556-40b8-890c-95ef7e28ca72 admin admin] PATH_INFO: `/v3/projects/eb0582b461b44e1b84628f5c18e12483` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:37.233630 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835f8b01-4556-40b8-890c-95ef7e28ca72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb0582b461b44e1b84628f5c18e12483)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:37.235582 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835f8b01-4556-40b8-890c-95ef7e28ca72 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:37.306781 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-835f8b01-4556-40b8-890c-95ef7e28ca72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb0582b461b44e1b84628f5c18e12483', 'request_id': 'req-835f8b01-4556-40b8-890c-95ef7e28ca72'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:37.316469 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-835f8b01-4556-40b8-890c-95ef7e28ca72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb0582b461b44e1b84628f5c18e12483', 'request_id': 'req-835f8b01-4556-40b8-890c-95ef7e28ca72'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:37.328248 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-835f8b01-4556-40b8-890c-95ef7e28ca72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb0582b461b44e1b84628f5c18e12483', 'request_id': 'req-835f8b01-4556-40b8-890c-95ef7e28ca72'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:37.342654 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-835f8b01-4556-40b8-890c-95ef7e28ca72 admin admin] The token cache is being invalidate because project eb0582b461b44e1b84628f5c18e12483 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:42:37.343111 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-835f8b01-4556-40b8-890c-95ef7e28ca72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:37.344696 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-835f8b01-4556-40b8-890c-95ef7e28ca72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:37.346586 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 941/1895] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:42:37 2022] DELETE /identity/v3/projects/eb0582b461b44e1b84628f5c18e12483 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:37.351385 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede784ac-cc1e-43b9-932b-733b2ffc5df2 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:37.351855 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede784ac-cc1e-43b9-932b-733b2ffc5df2 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:37.352132 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede784ac-cc1e-43b9-932b-733b2ffc5df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:37.380706 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-ede784ac-cc1e-43b9-932b-733b2ffc5df2 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:37.382126 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ede784ac-cc1e-43b9-932b-733b2ffc5df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:37.450584 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 955/1896] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:42:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:37.458717 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7387db90-80c4-41cd-9cb5-0ee5f1968737 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:37.459207 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7387db90-80c4-41cd-9cb5-0ee5f1968737 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:37.459301 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7387db90-80c4-41cd-9cb5-0ee5f1968737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:37.479388 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-7387db90-80c4-41cd-9cb5-0ee5f1968737 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:37.480597 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7387db90-80c4-41cd-9cb5-0ee5f1968737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:37.488524 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 942/1897] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:42:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:37.493068 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede784ac-cc1e-43b9-932b-733b2ffc5df2 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:37.502974 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede784ac-cc1e-43b9-932b-733b2ffc5df2 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:37.508013 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60419cea-150e-4276-9a6a-d1d0c1fd320d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:37.508649 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60419cea-150e-4276-9a6a-d1d0c1fd320d admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:37.508930 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60419cea-150e-4276-9a6a-d1d0c1fd320d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:37.509318 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60419cea-150e-4276-9a6a-d1d0c1fd320d admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:37.510505 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60419cea-150e-4276-9a6a-d1d0c1fd320d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:42:37.510892 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60419cea-150e-4276-9a6a-d1d0c1fd320d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:37.512479 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60419cea-150e-4276-9a6a-d1d0c1fd320d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:37.528538 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 956/1898] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:42:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:42:37.529733 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7387db90-80c4-41cd-9cb5-0ee5f1968737 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:37.539705 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7387db90-80c4-41cd-9cb5-0ee5f1968737 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:37.545433 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3b79d1-bf5a-46ac-980f-4f2eb156a48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:37.546071 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b79d1-bf5a-46ac-980f-4f2eb156a48d admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:37.546332 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b79d1-bf5a-46ac-980f-4f2eb156a48d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:37.546605 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b79d1-bf5a-46ac-980f-4f2eb156a48d admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:37.547247 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3b79d1-bf5a-46ac-980f-4f2eb156a48d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:37.549142 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3b79d1-bf5a-46ac-980f-4f2eb156a48d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:37.581305 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 943/1899] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:42:37 2022] POST /identity/v3/projects => generated 401 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:37.582634 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60419cea-150e-4276-9a6a-d1d0c1fd320d admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:37.594181 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60419cea-150e-4276-9a6a-d1d0c1fd320d admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:37.675444 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e767a7-7251-49ad-bef6-06dd697d04c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:37.676071 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e767a7-7251-49ad-bef6-06dd697d04c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:37.676362 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e767a7-7251-49ad-bef6-06dd697d04c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:37.676632 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e767a7-7251-49ad-bef6-06dd697d04c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:37.677403 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e767a7-7251-49ad-bef6-06dd697d04c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:37.679506 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e767a7-7251-49ad-bef6-06dd697d04c4 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:37.702524 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 957/1900] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:42:37 2022] POST /identity/v3/users => generated 301 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:37.712681 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3b79d1-bf5a-46ac-980f-4f2eb156a48d admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:37.724765 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3b79d1-bf5a-46ac-980f-4f2eb156a48d admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:37.729416 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74ab95-983f-47f2-9733-2d02405f4379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:37.730046 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74ab95-983f-47f2-9733-2d02405f4379 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:37.730325 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74ab95-983f-47f2-9733-2d02405f4379 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:37.730593 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74ab95-983f-47f2-9733-2d02405f4379 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:37.731329 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb74ab95-983f-47f2-9733-2d02405f4379 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:37.732969 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb74ab95-983f-47f2-9733-2d02405f4379 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:37.747010 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 944/1901] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:37.748203 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e767a7-7251-49ad-bef6-06dd697d04c4 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:37.758709 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e767a7-7251-49ad-bef6-06dd697d04c4 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:37.766626 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afa7e82-e759-4fed-9893-49c52b54fc99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:37.767753 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afa7e82-e759-4fed-9893-49c52b54fc99 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:37.768206 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afa7e82-e759-4fed-9893-49c52b54fc99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:37.768551 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afa7e82-e759-4fed-9893-49c52b54fc99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:37.769492 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afa7e82-e759-4fed-9893-49c52b54fc99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:37.771884 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afa7e82-e759-4fed-9893-49c52b54fc99 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:37.783395 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 958/1902] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:37.787602 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74ab95-983f-47f2-9733-2d02405f4379 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:37.797651 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74ab95-983f-47f2-9733-2d02405f4379 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:37.802402 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55982f5c-f293-4558-a400-78a16b06f98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:37.803109 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55982f5c-f293-4558-a400-78a16b06f98f admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:37.803410 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55982f5c-f293-4558-a400-78a16b06f98f admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:37.804165 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55982f5c-f293-4558-a400-78a16b06f98f admin admin] PATH_INFO: `/v3/projects/bc27fcf092f144bab0f22492a053618c/users/77cb0df344a04162ad0e701481832a5a/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:37.820317 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55982f5c-f293-4558-a400-78a16b06f98f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc27fcf092f144bab0f22492a053618c, user_id=77cb0df344a04162ad0e701481832a5a, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:37.823524 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55982f5c-f293-4558-a400-78a16b06f98f admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:37.836647 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 945/1903] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:37 2022] PUT /identity/v3/projects/bc27fcf092f144bab0f22492a053618c/users/77cb0df344a04162ad0e701481832a5a/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:42:37.848181 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c260d54-b2f7-40dc-b52a-d7fc0a091157 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:37.848354 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c260d54-b2f7-40dc-b52a-d7fc0a091157 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:37.848525 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c260d54-b2f7-40dc-b52a-d7fc0a091157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:37.877396 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-8c260d54-b2f7-40dc-b52a-d7fc0a091157 None None] MFA Rules not processed for user `77cb0df344a04162ad0e701481832a5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:37.878717 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8c260d54-b2f7-40dc-b52a-d7fc0a091157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:37.940712 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:37.969247 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 959/1904] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:42:37 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:37.973659 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55982f5c-f293-4558-a400-78a16b06f98f admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:38.039189 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:38.058787 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55982f5c-f293-4558-a400-78a16b06f98f admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:38.064505 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f641b277-2668-4c98-930e-c15cbbfead72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:38.065392 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f641b277-2668-4c98-930e-c15cbbfead72 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:38.065763 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f641b277-2668-4c98-930e-c15cbbfead72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:38.066253 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f641b277-2668-4c98-930e-c15cbbfead72 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:38.067247 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f641b277-2668-4c98-930e-c15cbbfead72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:38.069475 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f641b277-2668-4c98-930e-c15cbbfead72 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:38.097971 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 946/1905] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:42:37 2022] POST /identity/v3/projects => generated 401 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:38.102628 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c260d54-b2f7-40dc-b52a-d7fc0a091157 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:38.115081 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c260d54-b2f7-40dc-b52a-d7fc0a091157 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:38.181679 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c5d9a-f7b1-4715-8daf-e18d4117e44f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:38.183254 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c5d9a-f7b1-4715-8daf-e18d4117e44f admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:38.183254 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c5d9a-f7b1-4715-8daf-e18d4117e44f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:38.183512 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c5d9a-f7b1-4715-8daf-e18d4117e44f admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:38.184238 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3c5d9a-f7b1-4715-8daf-e18d4117e44f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:38.187004 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3c5d9a-f7b1-4715-8daf-e18d4117e44f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:38.224046 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 960/1906] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:42:38 2022] POST /identity/v3/users => generated 307 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:38.224678 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f641b277-2668-4c98-930e-c15cbbfead72 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:38.258628 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f641b277-2668-4c98-930e-c15cbbfead72 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:38.264691 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf00a62-bf12-49e0-a274-2d0a6548421f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:38.266153 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf00a62-bf12-49e0-a274-2d0a6548421f admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:38.266671 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf00a62-bf12-49e0-a274-2d0a6548421f admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:38.266970 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf00a62-bf12-49e0-a274-2d0a6548421f admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:38.267631 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf00a62-bf12-49e0-a274-2d0a6548421f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:38.269899 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf00a62-bf12-49e0-a274-2d0a6548421f admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:38.303237 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 947/1907] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:38 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:42:38.306255 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c5d9a-f7b1-4715-8daf-e18d4117e44f admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:38.322220 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c5d9a-f7b1-4715-8daf-e18d4117e44f admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:38.330399 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd7bcde-0d76-479d-9d0b-9ae1863f3acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:38.331161 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7bcde-0d76-479d-9d0b-9ae1863f3acc admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:38.331634 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7bcde-0d76-479d-9d0b-9ae1863f3acc admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:38.331836 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7bcde-0d76-479d-9d0b-9ae1863f3acc admin admin] PATH_INFO: `/v3/projects/2d6aab9c0af94481bdae8df7077f1e4f/users/3cbf6e163d604f5998704d10c45d0692/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:38.368064 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd7bcde-0d76-479d-9d0b-9ae1863f3acc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d6aab9c0af94481bdae8df7077f1e4f, user_id=3cbf6e163d604f5998704d10c45d0692, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:38.372812 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd7bcde-0d76-479d-9d0b-9ae1863f3acc admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:38.399192 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf00a62-bf12-49e0-a274-2d0a6548421f admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:38.406372 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 961/1908] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:38 2022] PUT /identity/v3/projects/2d6aab9c0af94481bdae8df7077f1e4f/users/3cbf6e163d604f5998704d10c45d0692/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:38.413118 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf00a62-bf12-49e0-a274-2d0a6548421f admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:38.548974 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836c0705-8fb8-4157-933b-b0b50d479d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:38.549720 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c0705-8fb8-4157-933b-b0b50d479d30 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:38.550040 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c0705-8fb8-4157-933b-b0b50d479d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:38.550362 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c0705-8fb8-4157-933b-b0b50d479d30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:38.551602 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836c0705-8fb8-4157-933b-b0b50d479d30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:38.554198 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836c0705-8fb8-4157-933b-b0b50d479d30 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:38.594704 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd7bcde-0d76-479d-9d0b-9ae1863f3acc admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:38.599018 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 948/1909] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:38 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:38.621428 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd7bcde-0d76-479d-9d0b-9ae1863f3acc admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:38.627495 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb87dc08-6b4d-4280-9797-c9f0c6f2ef45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:38.628420 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb87dc08-6b4d-4280-9797-c9f0c6f2ef45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:38.628799 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb87dc08-6b4d-4280-9797-c9f0c6f2ef45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:38.629324 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb87dc08-6b4d-4280-9797-c9f0c6f2ef45 admin admin] PATH_INFO: `/v3/projects/2d6aab9c0af94481bdae8df7077f1e4f/users/3cbf6e163d604f5998704d10c45d0692/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:38.634118 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb87dc08-6b4d-4280-9797-c9f0c6f2ef45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d6aab9c0af94481bdae8df7077f1e4f, user_id=3cbf6e163d604f5998704d10c45d0692, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:38.637144 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb87dc08-6b4d-4280-9797-c9f0c6f2ef45 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:38.662765 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ab472-4a56-49b0-8ef5-8ec18623c530 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:38.663232 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ab472-4a56-49b0-8ef5-8ec18623c530 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:38.663581 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ab472-4a56-49b0-8ef5-8ec18623c530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:38.666952 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 962/1910] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:38 2022] PUT /identity/v3/projects/2d6aab9c0af94481bdae8df7077f1e4f/users/3cbf6e163d604f5998704d10c45d0692/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:38.699599 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-d63ab472-4a56-49b0-8ef5-8ec18623c530 None None] MFA Rules not processed for user `3cbf6e163d604f5998704d10c45d0692`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:38.701369 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d63ab472-4a56-49b0-8ef5-8ec18623c530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:38.788865 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 949/1911] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:42:38 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:38.800691 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb87dc08-6b4d-4280-9797-c9f0c6f2ef45 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:38.802717 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-fb87dc08-6b4d-4280-9797-c9f0c6f2ef45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:38.880498 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb87dc08-6b4d-4280-9797-c9f0c6f2ef45 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:38.884495 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35bdcbe-bc0c-4307-b85c-31cd2d10687f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:38.885782 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35bdcbe-bc0c-4307-b85c-31cd2d10687f service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:38.886180 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35bdcbe-bc0c-4307-b85c-31cd2d10687f service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:38.886641 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35bdcbe-bc0c-4307-b85c-31cd2d10687f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:38.896187 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35bdcbe-bc0c-4307-b85c-31cd2d10687f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:38.897619 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35bdcbe-bc0c-4307-b85c-31cd2d10687f service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:38.909954 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 963/1912] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:38 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:42:38.989043 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63ab472-4a56-49b0-8ef5-8ec18623c530 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:38.999276 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63ab472-4a56-49b0-8ef5-8ec18623c530 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:39.003358 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef86d06-93ce-46f3-8e00-96584c236048 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:39.004147 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef86d06-93ce-46f3-8e00-96584c236048 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:39.004501 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef86d06-93ce-46f3-8e00-96584c236048 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:39.004837 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef86d06-93ce-46f3-8e00-96584c236048 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:39.058841 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:39.073975 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef86d06-93ce-46f3-8e00-96584c236048 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:39.075532 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef86d06-93ce-46f3-8e00-96584c236048 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:39.085493 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 950/1913] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:38 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:42:39.111135 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35bdcbe-bc0c-4307-b85c-31cd2d10687f service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:39.112544 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b35bdcbe-bc0c-4307-b85c-31cd2d10687f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:39.160407 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:39.181302 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35bdcbe-bc0c-4307-b85c-31cd2d10687f service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:39.186022 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0636872e-c6bc-40b6-8abd-1f959f1dbaa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:39.186655 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636872e-c6bc-40b6-8abd-1f959f1dbaa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:39.186933 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636872e-c6bc-40b6-8abd-1f959f1dbaa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:39.187207 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636872e-c6bc-40b6-8abd-1f959f1dbaa3 admin admin] PATH_INFO: `/v3/users/1c5eabeda8234861a28d38cb3839bec6` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:39.188314 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0636872e-c6bc-40b6-8abd-1f959f1dbaa3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c5eabeda8234861a28d38cb3839bec6)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:39.190300 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0636872e-c6bc-40b6-8abd-1f959f1dbaa3 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:39.240151 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-0636872e-c6bc-40b6-8abd-1f959f1dbaa3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c5eabeda8234861a28d38cb3839bec6'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:39.240810 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-0636872e-c6bc-40b6-8abd-1f959f1dbaa3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c5eabeda8234861a28d38cb3839bec6'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:39.255691 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-0636872e-c6bc-40b6-8abd-1f959f1dbaa3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c5eabeda8234861a28d38cb3839bec6'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:39.263232 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-0636872e-c6bc-40b6-8abd-1f959f1dbaa3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c5eabeda8234861a28d38cb3839bec6'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:39.270672 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-0636872e-c6bc-40b6-8abd-1f959f1dbaa3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c5eabeda8234861a28d38cb3839bec6'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:39.282197 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 964/1914] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:42:39 2022] DELETE /identity/v3/users/1c5eabeda8234861a28d38cb3839bec6 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:42:39.286762 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ca1eb-fc98-464e-9773-44ed6a17aa10 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:39.287055 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ca1eb-fc98-464e-9773-44ed6a17aa10 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:39.287345 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ca1eb-fc98-464e-9773-44ed6a17aa10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:39.306634 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-173ca1eb-fc98-464e-9773-44ed6a17aa10 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:39.308345 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-173ca1eb-fc98-464e-9773-44ed6a17aa10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:39.373283 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 951/1915] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:42:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:39.383837 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0636872e-c6bc-40b6-8abd-1f959f1dbaa3 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:39.386621 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0636872e-c6bc-40b6-8abd-1f959f1dbaa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:39.445755 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0636872e-c6bc-40b6-8abd-1f959f1dbaa3 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:39.450004 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46630110-ed2a-4868-bd4f-f46303bae160 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:39.450605 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46630110-ed2a-4868-bd4f-f46303bae160 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:39.450885 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46630110-ed2a-4868-bd4f-f46303bae160 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:39.451177 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46630110-ed2a-4868-bd4f-f46303bae160 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:39.460013 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46630110-ed2a-4868-bd4f-f46303bae160 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:39.461427 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46630110-ed2a-4868-bd4f-f46303bae160 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:39.470675 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 965/1916] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:42:39.743220 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173ca1eb-fc98-464e-9773-44ed6a17aa10 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:39.745826 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-173ca1eb-fc98-464e-9773-44ed6a17aa10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:39.763616 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173ca1eb-fc98-464e-9773-44ed6a17aa10 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:39.770155 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7892762-a0bc-48c5-959f-552ee3039aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:39.770792 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7892762-a0bc-48c5-959f-552ee3039aeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:39.771070 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7892762-a0bc-48c5-959f-552ee3039aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:39.771346 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7892762-a0bc-48c5-959f-552ee3039aeb admin admin] PATH_INFO: `/v3/projects/170cac2f0a4a429da386ddaa0642df8c` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:39.772461 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7892762-a0bc-48c5-959f-552ee3039aeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=170cac2f0a4a429da386ddaa0642df8c)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:39.774415 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7892762-a0bc-48c5-959f-552ee3039aeb admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:39.824743 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a7892762-a0bc-48c5-959f-552ee3039aeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '170cac2f0a4a429da386ddaa0642df8c', 'request_id': 'req-a7892762-a0bc-48c5-959f-552ee3039aeb'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:39.832541 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a7892762-a0bc-48c5-959f-552ee3039aeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '170cac2f0a4a429da386ddaa0642df8c', 'request_id': 'req-a7892762-a0bc-48c5-959f-552ee3039aeb'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:39.838616 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a7892762-a0bc-48c5-959f-552ee3039aeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '170cac2f0a4a429da386ddaa0642df8c', 'request_id': 'req-a7892762-a0bc-48c5-959f-552ee3039aeb'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:39.849354 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a7892762-a0bc-48c5-959f-552ee3039aeb admin admin] The token cache is being invalidate because project 170cac2f0a4a429da386ddaa0642df8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:42:39.849636 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a7892762-a0bc-48c5-959f-552ee3039aeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:39.850285 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a7892762-a0bc-48c5-959f-552ee3039aeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:39.851748 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 952/1917] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:42:39 2022] DELETE /identity/v3/projects/170cac2f0a4a429da386ddaa0642df8c => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:39.856165 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46630110-ed2a-4868-bd4f-f46303bae160 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:39.865727 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46630110-ed2a-4868-bd4f-f46303bae160 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:39.937388 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ff0a4-ec7d-45cb-8af4-1c955e382f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:39.938850 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ff0a4-ec7d-45cb-8af4-1c955e382f13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:39.939695 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ff0a4-ec7d-45cb-8af4-1c955e382f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:39.940027 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ff0a4-ec7d-45cb-8af4-1c955e382f13 admin admin] PATH_INFO: `/v3/users/15405b3ed20b4f749d30755f00665e73` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:39.941657 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0ff0a4-ec7d-45cb-8af4-1c955e382f13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15405b3ed20b4f749d30755f00665e73)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:39.945114 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0ff0a4-ec7d-45cb-8af4-1c955e382f13 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:39.951471 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:40.005885 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2f0ff0a4-ec7d-45cb-8af4-1c955e382f13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15405b3ed20b4f749d30755f00665e73'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:40.006695 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2f0ff0a4-ec7d-45cb-8af4-1c955e382f13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15405b3ed20b4f749d30755f00665e73'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:40.019511 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2f0ff0a4-ec7d-45cb-8af4-1c955e382f13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15405b3ed20b4f749d30755f00665e73'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:40.028849 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2f0ff0a4-ec7d-45cb-8af4-1c955e382f13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15405b3ed20b4f749d30755f00665e73'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:40.036460 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2f0ff0a4-ec7d-45cb-8af4-1c955e382f13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15405b3ed20b4f749d30755f00665e73'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:40.043178 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 966/1918] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:42:39 2022] DELETE /identity/v3/users/15405b3ed20b4f749d30755f00665e73 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:40.332405 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7892762-a0bc-48c5-959f-552ee3039aeb admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:40.333924 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a7892762-a0bc-48c5-959f-552ee3039aeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:40.351375 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7892762-a0bc-48c5-959f-552ee3039aeb admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:40.424094 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3748749-c99d-4339-884e-cfdd4955a62a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:40.426452 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3748749-c99d-4339-884e-cfdd4955a62a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:40.427551 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3748749-c99d-4339-884e-cfdd4955a62a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:40.428524 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3748749-c99d-4339-884e-cfdd4955a62a admin admin] PATH_INFO: `/v3/projects/19605a281cc340069680d5a73ec08e7a` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:40.430074 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3748749-c99d-4339-884e-cfdd4955a62a admin admin] RBAC: Authorizing `identity:delete_project(project_id=19605a281cc340069680d5a73ec08e7a)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:40.432693 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3748749-c99d-4339-884e-cfdd4955a62a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:40.467722 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:40.535430 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d3748749-c99d-4339-884e-cfdd4955a62a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19605a281cc340069680d5a73ec08e7a', 'request_id': 'req-d3748749-c99d-4339-884e-cfdd4955a62a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:40.545492 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d3748749-c99d-4339-884e-cfdd4955a62a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19605a281cc340069680d5a73ec08e7a', 'request_id': 'req-d3748749-c99d-4339-884e-cfdd4955a62a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:40.554544 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d3748749-c99d-4339-884e-cfdd4955a62a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19605a281cc340069680d5a73ec08e7a', 'request_id': 'req-d3748749-c99d-4339-884e-cfdd4955a62a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:40.572001 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d3748749-c99d-4339-884e-cfdd4955a62a admin admin] The token cache is being invalidate because project 19605a281cc340069680d5a73ec08e7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:42:40.572831 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d3748749-c99d-4339-884e-cfdd4955a62a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:40.573503 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d3748749-c99d-4339-884e-cfdd4955a62a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:40.575078 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 953/1919] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:42:40 2022] DELETE /identity/v3/projects/19605a281cc340069680d5a73ec08e7a => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:40.579156 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ff0a4-ec7d-45cb-8af4-1c955e382f13 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:40.580616 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2f0ff0a4-ec7d-45cb-8af4-1c955e382f13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:40.594183 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ff0a4-ec7d-45cb-8af4-1c955e382f13 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:40.662621 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d41f1e3-67e9-4b07-8609-45ea9e3ebcfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:40.663269 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41f1e3-67e9-4b07-8609-45ea9e3ebcfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:40.663550 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41f1e3-67e9-4b07-8609-45ea9e3ebcfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:40.663880 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41f1e3-67e9-4b07-8609-45ea9e3ebcfe admin admin] PATH_INFO: `/v3/users/3348c0100afa4fc19406ec9690c89572` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:40.665042 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d41f1e3-67e9-4b07-8609-45ea9e3ebcfe admin admin] RBAC: Authorizing `identity:delete_user(user_id=3348c0100afa4fc19406ec9690c89572)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:40.667171 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d41f1e3-67e9-4b07-8609-45ea9e3ebcfe admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:40.731228 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-8d41f1e3-67e9-4b07-8609-45ea9e3ebcfe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3348c0100afa4fc19406ec9690c89572'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:40.732051 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-8d41f1e3-67e9-4b07-8609-45ea9e3ebcfe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3348c0100afa4fc19406ec9690c89572'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:40.753032 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-8d41f1e3-67e9-4b07-8609-45ea9e3ebcfe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3348c0100afa4fc19406ec9690c89572'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:40.777727 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-8d41f1e3-67e9-4b07-8609-45ea9e3ebcfe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3348c0100afa4fc19406ec9690c89572'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:40.788270 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-8d41f1e3-67e9-4b07-8609-45ea9e3ebcfe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3348c0100afa4fc19406ec9690c89572'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:40.796060 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 967/1920] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:42:40 2022] DELETE /identity/v3/users/3348c0100afa4fc19406ec9690c89572 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:41.136929 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3748749-c99d-4339-884e-cfdd4955a62a admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:41.138489 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d3748749-c99d-4339-884e-cfdd4955a62a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:41.169227 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3748749-c99d-4339-884e-cfdd4955a62a admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:41.227544 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787c2ef2-424e-4b15-b0cd-b80c32297653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:41.228222 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787c2ef2-424e-4b15-b0cd-b80c32297653 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:41.229136 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787c2ef2-424e-4b15-b0cd-b80c32297653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:41.229425 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787c2ef2-424e-4b15-b0cd-b80c32297653 admin admin] PATH_INFO: `/v3/projects/b972ae166c384e7fae3a4b7040c1a7a7` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:41.230696 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787c2ef2-424e-4b15-b0cd-b80c32297653 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b972ae166c384e7fae3a4b7040c1a7a7)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:41.232715 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787c2ef2-424e-4b15-b0cd-b80c32297653 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:41.303091 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-787c2ef2-424e-4b15-b0cd-b80c32297653 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b972ae166c384e7fae3a4b7040c1a7a7', 'request_id': 'req-787c2ef2-424e-4b15-b0cd-b80c32297653'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:41.315217 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-787c2ef2-424e-4b15-b0cd-b80c32297653 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b972ae166c384e7fae3a4b7040c1a7a7', 'request_id': 'req-787c2ef2-424e-4b15-b0cd-b80c32297653'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:41.323412 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-787c2ef2-424e-4b15-b0cd-b80c32297653 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b972ae166c384e7fae3a4b7040c1a7a7', 'request_id': 'req-787c2ef2-424e-4b15-b0cd-b80c32297653'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:41.339724 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-787c2ef2-424e-4b15-b0cd-b80c32297653 admin admin] The token cache is being invalidate because project b972ae166c384e7fae3a4b7040c1a7a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:42:41.340115 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-787c2ef2-424e-4b15-b0cd-b80c32297653 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:41.341421 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-787c2ef2-424e-4b15-b0cd-b80c32297653 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:41.343542 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 954/1921] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:42:41 2022] DELETE /identity/v3/projects/b972ae166c384e7fae3a4b7040c1a7a7 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:41.349370 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8adaa5-5049-4b1e-a4bf-0bf5d238d0bb None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:41.349921 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8adaa5-5049-4b1e-a4bf-0bf5d238d0bb None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:41.349921 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8adaa5-5049-4b1e-a4bf-0bf5d238d0bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:41.357927 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:41.373498 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-9b8adaa5-5049-4b1e-a4bf-0bf5d238d0bb None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:41.375303 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9b8adaa5-5049-4b1e-a4bf-0bf5d238d0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:41.436669 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 968/1922] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:42:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:41.445278 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02efc16b-1382-44aa-abe1-7f60f5789227 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:41.445568 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02efc16b-1382-44aa-abe1-7f60f5789227 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:41.445863 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02efc16b-1382-44aa-abe1-7f60f5789227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:41.466539 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-02efc16b-1382-44aa-abe1-7f60f5789227 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:41.467776 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-02efc16b-1382-44aa-abe1-7f60f5789227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:41.479194 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 955/1923] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:42:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:41.488279 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8adaa5-5049-4b1e-a4bf-0bf5d238d0bb None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:41.498152 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8adaa5-5049-4b1e-a4bf-0bf5d238d0bb None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:41.504064 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86cde17-342b-45d1-bdbe-02db511b5969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:41.505061 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86cde17-342b-45d1-bdbe-02db511b5969 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:41.505443 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86cde17-342b-45d1-bdbe-02db511b5969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:41.505814 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86cde17-342b-45d1-bdbe-02db511b5969 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:41.507109 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86cde17-342b-45d1-bdbe-02db511b5969 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:42:41.507109 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86cde17-342b-45d1-bdbe-02db511b5969 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:41.509115 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86cde17-342b-45d1-bdbe-02db511b5969 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:41.524001 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 969/1924] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:42:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:42:41.528945 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02efc16b-1382-44aa-abe1-7f60f5789227 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:41.539725 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02efc16b-1382-44aa-abe1-7f60f5789227 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:41.545544 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa83475c-c4ba-4800-97d9-fd8e33950371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:41.546314 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83475c-c4ba-4800-97d9-fd8e33950371 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:41.546778 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83475c-c4ba-4800-97d9-fd8e33950371 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:41.547140 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83475c-c4ba-4800-97d9-fd8e33950371 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:41.547866 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa83475c-c4ba-4800-97d9-fd8e33950371 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:41.549880 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa83475c-c4ba-4800-97d9-fd8e33950371 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:41.578805 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 956/1925] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:42:41 2022] POST /identity/v3/projects => generated 391 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:41.582917 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86cde17-342b-45d1-bdbe-02db511b5969 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:41.594451 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86cde17-342b-45d1-bdbe-02db511b5969 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:41.654410 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71869d7-25e6-47f4-a53d-56315bc07e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:41.655034 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71869d7-25e6-47f4-a53d-56315bc07e3b admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:41.655320 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71869d7-25e6-47f4-a53d-56315bc07e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:41.655598 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71869d7-25e6-47f4-a53d-56315bc07e3b admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:41.656230 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71869d7-25e6-47f4-a53d-56315bc07e3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:41.658273 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71869d7-25e6-47f4-a53d-56315bc07e3b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:41.684874 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 970/1926] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:42:41 2022] POST /identity/v3/users => generated 296 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:42:41.686443 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa83475c-c4ba-4800-97d9-fd8e33950371 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:41.691879 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:41.698331 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa83475c-c4ba-4800-97d9-fd8e33950371 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:41.705265 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83e2d5d-2eb1-437f-9901-03d5282b3ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:41.706096 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e2d5d-2eb1-437f-9901-03d5282b3ce0 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:41.706654 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e2d5d-2eb1-437f-9901-03d5282b3ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:41.706927 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e2d5d-2eb1-437f-9901-03d5282b3ce0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:41.707612 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83e2d5d-2eb1-437f-9901-03d5282b3ce0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:41.709249 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83e2d5d-2eb1-437f-9901-03d5282b3ce0 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:41.735523 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 957/1927] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:41 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:41.740591 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71869d7-25e6-47f4-a53d-56315bc07e3b admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:41.752904 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71869d7-25e6-47f4-a53d-56315bc07e3b admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:41.759428 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f6414-26bc-4d25-a892-fbb067492e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:41.760174 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f6414-26bc-4d25-a892-fbb067492e35 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:41.760516 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f6414-26bc-4d25-a892-fbb067492e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:41.760852 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f6414-26bc-4d25-a892-fbb067492e35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:41.761608 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80f6414-26bc-4d25-a892-fbb067492e35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:41.763348 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80f6414-26bc-4d25-a892-fbb067492e35 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:41.774341 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 971/1928] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:41.779213 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83e2d5d-2eb1-437f-9901-03d5282b3ce0 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:41.790537 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83e2d5d-2eb1-437f-9901-03d5282b3ce0 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:41.797808 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90165910-9c0e-4a51-8007-d9835800d1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:41.798816 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90165910-9c0e-4a51-8007-d9835800d1d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:41.799203 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90165910-9c0e-4a51-8007-d9835800d1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:41.799790 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90165910-9c0e-4a51-8007-d9835800d1d4 admin admin] PATH_INFO: `/v3/projects/f825cc3d4dfa417f8fd8f8a10ec2c248/users/a5055d2bc3ce498a8a47863ce51f9975/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:41.819600 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90165910-9c0e-4a51-8007-d9835800d1d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f825cc3d4dfa417f8fd8f8a10ec2c248, user_id=a5055d2bc3ce498a8a47863ce51f9975, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:41.822427 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90165910-9c0e-4a51-8007-d9835800d1d4 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:41.841744 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 958/1929] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:41 2022] PUT /identity/v3/projects/f825cc3d4dfa417f8fd8f8a10ec2c248/users/a5055d2bc3ce498a8a47863ce51f9975/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:41.856148 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d078bbb-017b-4820-9b89-b850315e9e15 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:41.856434 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d078bbb-017b-4820-9b89-b850315e9e15 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:41.856794 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d078bbb-017b-4820-9b89-b850315e9e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:41.892995 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-9d078bbb-017b-4820-9b89-b850315e9e15 None None] MFA Rules not processed for user `a5055d2bc3ce498a8a47863ce51f9975`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:41.895087 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9d078bbb-017b-4820-9b89-b850315e9e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:41.987355 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:41.993962 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 972/1930] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:42:41 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:41.998854 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90165910-9c0e-4a51-8007-d9835800d1d4 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:42.096729 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90165910-9c0e-4a51-8007-d9835800d1d4 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:42.101753 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb6346b-857a-44ba-b6ef-b9f50c0ebcf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:42.102459 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6346b-857a-44ba-b6ef-b9f50c0ebcf6 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:42.102831 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6346b-857a-44ba-b6ef-b9f50c0ebcf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:42.103286 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6346b-857a-44ba-b6ef-b9f50c0ebcf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:42.105062 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb6346b-857a-44ba-b6ef-b9f50c0ebcf6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:42.107085 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb6346b-857a-44ba-b6ef-b9f50c0ebcf6 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:42.142761 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 959/1931] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:42:41 2022] POST /identity/v3/projects => generated 393 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:42.143485 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d078bbb-017b-4820-9b89-b850315e9e15 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:42.160589 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d078bbb-017b-4820-9b89-b850315e9e15 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:42.237040 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d42785-6a80-4f81-a67e-4b673ce103d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:42.238007 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d42785-6a80-4f81-a67e-4b673ce103d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:42.238294 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d42785-6a80-4f81-a67e-4b673ce103d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:42.238563 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d42785-6a80-4f81-a67e-4b673ce103d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:42.239355 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d42785-6a80-4f81-a67e-4b673ce103d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:42.241366 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d42785-6a80-4f81-a67e-4b673ce103d2 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:42.274602 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 973/1932] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:42:42 2022] POST /identity/v3/users => generated 303 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:42.278943 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb6346b-857a-44ba-b6ef-b9f50c0ebcf6 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:42.289781 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb6346b-857a-44ba-b6ef-b9f50c0ebcf6 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:42.296491 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd07d04-c6e1-4ccf-9add-f21ec96e2ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:42.297139 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd07d04-c6e1-4ccf-9add-f21ec96e2ed1 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:42.297419 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd07d04-c6e1-4ccf-9add-f21ec96e2ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:42.297686 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd07d04-c6e1-4ccf-9add-f21ec96e2ed1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:42.298407 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd07d04-c6e1-4ccf-9add-f21ec96e2ed1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:42.300163 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd07d04-c6e1-4ccf-9add-f21ec96e2ed1 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:42.312645 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 960/1933] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:42:42.317129 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d42785-6a80-4f81-a67e-4b673ce103d2 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:42.328169 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d42785-6a80-4f81-a67e-4b673ce103d2 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:42.332913 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b013da-e72a-4a79-a274-dc7397af51b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:42.333585 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b013da-e72a-4a79-a274-dc7397af51b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:42.333896 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b013da-e72a-4a79-a274-dc7397af51b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:42.334171 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b013da-e72a-4a79-a274-dc7397af51b4 admin admin] PATH_INFO: `/v3/projects/18c7cd1babcf409094c60f6af3973bc7/users/817e2da961a84fe0921cac5d3e84943f/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:42.349206 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b013da-e72a-4a79-a274-dc7397af51b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18c7cd1babcf409094c60f6af3973bc7, user_id=817e2da961a84fe0921cac5d3e84943f, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:42.351966 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b013da-e72a-4a79-a274-dc7397af51b4 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:42.365261 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 974/1934] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:42 2022] PUT /identity/v3/projects/18c7cd1babcf409094c60f6af3973bc7/users/817e2da961a84fe0921cac5d3e84943f/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:42:42.370169 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd07d04-c6e1-4ccf-9add-f21ec96e2ed1 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:42.380458 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd07d04-c6e1-4ccf-9add-f21ec96e2ed1 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:42.440718 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758c1f42-841f-4cae-94e5-1dfc7912782f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:42.441344 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c1f42-841f-4cae-94e5-1dfc7912782f admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:42.441616 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c1f42-841f-4cae-94e5-1dfc7912782f admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:42.441900 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c1f42-841f-4cae-94e5-1dfc7912782f admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:42.442551 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758c1f42-841f-4cae-94e5-1dfc7912782f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:42.444172 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758c1f42-841f-4cae-94e5-1dfc7912782f admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:42.456206 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 961/1935] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:42.460619 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b013da-e72a-4a79-a274-dc7397af51b4 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:42.472543 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b013da-e72a-4a79-a274-dc7397af51b4 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:42.477988 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a918249e-db24-4e8d-acfe-341c4d136077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:42.478704 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a918249e-db24-4e8d-acfe-341c4d136077 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:42.479015 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a918249e-db24-4e8d-acfe-341c4d136077 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:42.479343 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a918249e-db24-4e8d-acfe-341c4d136077 admin admin] PATH_INFO: `/v3/projects/18c7cd1babcf409094c60f6af3973bc7/users/817e2da961a84fe0921cac5d3e84943f/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:42.481627 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a918249e-db24-4e8d-acfe-341c4d136077 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18c7cd1babcf409094c60f6af3973bc7, user_id=817e2da961a84fe0921cac5d3e84943f, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:42.484916 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a918249e-db24-4e8d-acfe-341c4d136077 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:42.498372 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 975/1936] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:42 2022] PUT /identity/v3/projects/18c7cd1babcf409094c60f6af3973bc7/users/817e2da961a84fe0921cac5d3e84943f/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:42.511329 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4728f98d-cb3f-4739-ab5f-031b20d09fec None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:42.511616 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4728f98d-cb3f-4739-ab5f-031b20d09fec None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:42.511873 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4728f98d-cb3f-4739-ab5f-031b20d09fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:42.539591 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-4728f98d-cb3f-4739-ab5f-031b20d09fec None None] MFA Rules not processed for user `817e2da961a84fe0921cac5d3e84943f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:42.540809 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4728f98d-cb3f-4739-ab5f-031b20d09fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:42.567313 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:42.612985 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 962/1937] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:42:42 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:42.624779 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a918249e-db24-4e8d-acfe-341c4d136077 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:42.626484 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a918249e-db24-4e8d-acfe-341c4d136077 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:42.683173 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a918249e-db24-4e8d-acfe-341c4d136077 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:42.686687 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b9a33c-a528-4b32-9975-3258f746860c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:42.687289 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b9a33c-a528-4b32-9975-3258f746860c service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:42.687569 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b9a33c-a528-4b32-9975-3258f746860c service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:42.687844 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b9a33c-a528-4b32-9975-3258f746860c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:42.697346 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b9a33c-a528-4b32-9975-3258f746860c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:42.698795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b9a33c-a528-4b32-9975-3258f746860c service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:42.708105 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 976/1938] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:42 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:42:42.759853 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4728f98d-cb3f-4739-ab5f-031b20d09fec None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:42.761300 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4728f98d-cb3f-4739-ab5f-031b20d09fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:42.774614 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4728f98d-cb3f-4739-ab5f-031b20d09fec None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:42.779363 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4279c5-98df-4ff5-aeb0-dc222f41021d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:42.780176 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4279c5-98df-4ff5-aeb0-dc222f41021d service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:42.780548 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4279c5-98df-4ff5-aeb0-dc222f41021d service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:42.780918 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4279c5-98df-4ff5-aeb0-dc222f41021d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:42.841041 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4279c5-98df-4ff5-aeb0-dc222f41021d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:42.842523 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4279c5-98df-4ff5-aeb0-dc222f41021d service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:42.851894 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 963/1939] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:42 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:42:43.422673 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b9a33c-a528-4b32-9975-3258f746860c service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:43.424090 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-86b9a33c-a528-4b32-9975-3258f746860c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:43.475953 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:43.490105 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b9a33c-a528-4b32-9975-3258f746860c service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:43.495487 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d8c45-5d33-4a3e-882c-c9e7fd1c45ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:43.496402 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d8c45-5d33-4a3e-882c-c9e7fd1c45ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:43.496705 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d8c45-5d33-4a3e-882c-c9e7fd1c45ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:43.497031 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d8c45-5d33-4a3e-882c-c9e7fd1c45ae admin admin] PATH_INFO: `/v3/users/77cb0df344a04162ad0e701481832a5a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:43.498284 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d8c45-5d33-4a3e-882c-c9e7fd1c45ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=77cb0df344a04162ad0e701481832a5a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:43.500493 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d8c45-5d33-4a3e-882c-c9e7fd1c45ae admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:43.546997 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-db3d8c45-5d33-4a3e-882c-c9e7fd1c45ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77cb0df344a04162ad0e701481832a5a'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:43.547686 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-db3d8c45-5d33-4a3e-882c-c9e7fd1c45ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77cb0df344a04162ad0e701481832a5a'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:43.559337 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-db3d8c45-5d33-4a3e-882c-c9e7fd1c45ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77cb0df344a04162ad0e701481832a5a'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:43.566338 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-db3d8c45-5d33-4a3e-882c-c9e7fd1c45ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77cb0df344a04162ad0e701481832a5a'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:43.573623 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-db3d8c45-5d33-4a3e-882c-c9e7fd1c45ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77cb0df344a04162ad0e701481832a5a'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:43.579581 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 977/1940] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:42:43 2022] DELETE /identity/v3/users/77cb0df344a04162ad0e701481832a5a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:43.584283 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86861af5-617e-4276-a228-386a72059fd0 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:43.584784 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86861af5-617e-4276-a228-386a72059fd0 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:43.584905 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86861af5-617e-4276-a228-386a72059fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:43.604267 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-86861af5-617e-4276-a228-386a72059fd0 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:43.605493 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-86861af5-617e-4276-a228-386a72059fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:43.665663 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 964/1941] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:42:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:43.675065 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d8c45-5d33-4a3e-882c-c9e7fd1c45ae admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:43.676483 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-db3d8c45-5d33-4a3e-882c-c9e7fd1c45ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:43.732951 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d8c45-5d33-4a3e-882c-c9e7fd1c45ae admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:43.736413 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9db1762-6825-4dd0-a027-ab1b9474bb20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:43.737005 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db1762-6825-4dd0-a027-ab1b9474bb20 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:43.737281 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db1762-6825-4dd0-a027-ab1b9474bb20 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:43.737549 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db1762-6825-4dd0-a027-ab1b9474bb20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:43.746708 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9db1762-6825-4dd0-a027-ab1b9474bb20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:43.748112 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9db1762-6825-4dd0-a027-ab1b9474bb20 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:43.757914 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 978/1942] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:42:43.952091 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae3305f-91d8-4e91-a836-0ae561d38458 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:43.952402 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae3305f-91d8-4e91-a836-0ae561d38458 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:43.952700 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae3305f-91d8-4e91-a836-0ae561d38458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:43.972337 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-1ae3305f-91d8-4e91-a836-0ae561d38458 None None] MFA Rules not processed for user `5558f9c2bb37403b9ff03f2be1aa3113`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:43.973554 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-1ae3305f-91d8-4e91-a836-0ae561d38458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:44.044391 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9db1762-6825-4dd0-a027-ab1b9474bb20 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:44.049806 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f9db1762-6825-4dd0-a027-ab1b9474bb20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:44.049975 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 965/1943] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:42:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:44.077589 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9db1762-6825-4dd0-a027-ab1b9474bb20 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:44.084847 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c5dcd9-ded5-4902-9f8c-045e6b5870be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:44.087405 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5dcd9-ded5-4902-9f8c-045e6b5870be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:44.088080 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5dcd9-ded5-4902-9f8c-045e6b5870be admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:44.088510 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5dcd9-ded5-4902-9f8c-045e6b5870be admin admin] PATH_INFO: `/v3/projects/bc27fcf092f144bab0f22492a053618c` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:44.089938 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c5dcd9-ded5-4902-9f8c-045e6b5870be admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc27fcf092f144bab0f22492a053618c)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:44.092368 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c5dcd9-ded5-4902-9f8c-045e6b5870be admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:44.149690 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-f2c5dcd9-ded5-4902-9f8c-045e6b5870be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc27fcf092f144bab0f22492a053618c', 'request_id': 'req-f2c5dcd9-ded5-4902-9f8c-045e6b5870be'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:44.156128 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-f2c5dcd9-ded5-4902-9f8c-045e6b5870be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc27fcf092f144bab0f22492a053618c', 'request_id': 'req-f2c5dcd9-ded5-4902-9f8c-045e6b5870be'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:44.167320 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-f2c5dcd9-ded5-4902-9f8c-045e6b5870be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc27fcf092f144bab0f22492a053618c', 'request_id': 'req-f2c5dcd9-ded5-4902-9f8c-045e6b5870be'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:44.174457 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:44.180333 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-f2c5dcd9-ded5-4902-9f8c-045e6b5870be admin admin] The token cache is being invalidate because project bc27fcf092f144bab0f22492a053618c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:42:44.180714 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-f2c5dcd9-ded5-4902-9f8c-045e6b5870be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:44.182237 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-f2c5dcd9-ded5-4902-9f8c-045e6b5870be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:44.184292 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 979/1944] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:42:44 2022] DELETE /identity/v3/projects/bc27fcf092f144bab0f22492a053618c => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:42:44.188428 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae3305f-91d8-4e91-a836-0ae561d38458 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:44.200476 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae3305f-91d8-4e91-a836-0ae561d38458 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:44.204401 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:44.288771 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25754d19-9900-4e3a-a924-5dc4053cba7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:44.290041 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25754d19-9900-4e3a-a924-5dc4053cba7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:44.290382 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25754d19-9900-4e3a-a924-5dc4053cba7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:44.290665 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25754d19-9900-4e3a-a924-5dc4053cba7f admin admin] PATH_INFO: `/v3/users/3cbf6e163d604f5998704d10c45d0692` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:44.292486 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25754d19-9900-4e3a-a924-5dc4053cba7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cbf6e163d604f5998704d10c45d0692)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:44.294586 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25754d19-9900-4e3a-a924-5dc4053cba7f admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:44.348540 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-25754d19-9900-4e3a-a924-5dc4053cba7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cbf6e163d604f5998704d10c45d0692'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:44.356741 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-25754d19-9900-4e3a-a924-5dc4053cba7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cbf6e163d604f5998704d10c45d0692'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:44.357442 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-25754d19-9900-4e3a-a924-5dc4053cba7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cbf6e163d604f5998704d10c45d0692'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:44.365707 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-25754d19-9900-4e3a-a924-5dc4053cba7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cbf6e163d604f5998704d10c45d0692'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:44.366407 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-25754d19-9900-4e3a-a924-5dc4053cba7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cbf6e163d604f5998704d10c45d0692'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:44.387352 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 966/1945] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:42:44 2022] DELETE /identity/v3/users/3cbf6e163d604f5998704d10c45d0692 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:44.896916 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c5dcd9-ded5-4902-9f8c-045e6b5870be admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:44.898532 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f2c5dcd9-ded5-4902-9f8c-045e6b5870be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:44.913328 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c5dcd9-ded5-4902-9f8c-045e6b5870be admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:44.985618 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc2392a-ba10-4ee9-a4c4-7044e965959d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:44.986887 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc2392a-ba10-4ee9-a4c4-7044e965959d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:44.987179 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc2392a-ba10-4ee9-a4c4-7044e965959d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:44.987460 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc2392a-ba10-4ee9-a4c4-7044e965959d admin admin] PATH_INFO: `/v3/projects/2d6aab9c0af94481bdae8df7077f1e4f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:44.988603 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc2392a-ba10-4ee9-a4c4-7044e965959d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d6aab9c0af94481bdae8df7077f1e4f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:44.991082 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc2392a-ba10-4ee9-a4c4-7044e965959d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:45.051648 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2fc2392a-ba10-4ee9-a4c4-7044e965959d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d6aab9c0af94481bdae8df7077f1e4f', 'request_id': 'req-2fc2392a-ba10-4ee9-a4c4-7044e965959d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:45.059203 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2fc2392a-ba10-4ee9-a4c4-7044e965959d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d6aab9c0af94481bdae8df7077f1e4f', 'request_id': 'req-2fc2392a-ba10-4ee9-a4c4-7044e965959d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:45.069338 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2fc2392a-ba10-4ee9-a4c4-7044e965959d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d6aab9c0af94481bdae8df7077f1e4f', 'request_id': 'req-2fc2392a-ba10-4ee9-a4c4-7044e965959d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:45.081323 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2fc2392a-ba10-4ee9-a4c4-7044e965959d admin admin] The token cache is being invalidate because project 2d6aab9c0af94481bdae8df7077f1e4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:42:45.081619 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2fc2392a-ba10-4ee9-a4c4-7044e965959d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:45.082286 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2fc2392a-ba10-4ee9-a4c4-7044e965959d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:45.084078 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 980/1946] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:42:44 2022] DELETE /identity/v3/projects/2d6aab9c0af94481bdae8df7077f1e4f => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:45.089488 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd55a5-04f7-44c4-9183-2c8533dc4d0d None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:45.090416 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd55a5-04f7-44c4-9183-2c8533dc4d0d None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:45.090954 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd55a5-04f7-44c4-9183-2c8533dc4d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:45.115655 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-84bd55a5-04f7-44c4-9183-2c8533dc4d0d None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:45.119998 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-84bd55a5-04f7-44c4-9183-2c8533dc4d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:45.207168 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 967/1947] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:42:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:45.217076 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4185296c-0b2f-4241-a99e-a6c7a11032c3 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:45.217360 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4185296c-0b2f-4241-a99e-a6c7a11032c3 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:45.217626 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4185296c-0b2f-4241-a99e-a6c7a11032c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:45.242240 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-4185296c-0b2f-4241-a99e-a6c7a11032c3 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:45.243461 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4185296c-0b2f-4241-a99e-a6c7a11032c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:45.253071 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 981/1948] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:42:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:45.262061 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd55a5-04f7-44c4-9183-2c8533dc4d0d None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:45.268237 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd55a5-04f7-44c4-9183-2c8533dc4d0d None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:45.273903 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a731d7a0-5eb1-4cee-a565-a778d2b0073a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:45.273903 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a731d7a0-5eb1-4cee-a565-a778d2b0073a admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:45.273903 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a731d7a0-5eb1-4cee-a565-a778d2b0073a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:45.274170 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a731d7a0-5eb1-4cee-a565-a778d2b0073a admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:45.274723 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a731d7a0-5eb1-4cee-a565-a778d2b0073a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:42:45.275019 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a731d7a0-5eb1-4cee-a565-a778d2b0073a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:45.277302 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a731d7a0-5eb1-4cee-a565-a778d2b0073a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:45.295723 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 968/1949] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:42:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:42:45.296353 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4185296c-0b2f-4241-a99e-a6c7a11032c3 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:45.306176 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4185296c-0b2f-4241-a99e-a6c7a11032c3 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:45.311678 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5757f58a-5f61-4f2a-ba7b-2c0edb864eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:45.312394 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757f58a-5f61-4f2a-ba7b-2c0edb864eed admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:45.312923 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757f58a-5f61-4f2a-ba7b-2c0edb864eed admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:45.313358 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757f58a-5f61-4f2a-ba7b-2c0edb864eed admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:45.314104 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5757f58a-5f61-4f2a-ba7b-2c0edb864eed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:45.316218 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5757f58a-5f61-4f2a-ba7b-2c0edb864eed admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:45.349903 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 982/1950] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:42:45 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:45.351487 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a731d7a0-5eb1-4cee-a565-a778d2b0073a admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:45.364852 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a731d7a0-5eb1-4cee-a565-a778d2b0073a admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:45.376656 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:45.437006 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be649d-8458-46df-a72d-c08a2549a00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:45.437698 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be649d-8458-46df-a72d-c08a2549a00d admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:45.438075 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be649d-8458-46df-a72d-c08a2549a00d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:45.438348 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be649d-8458-46df-a72d-c08a2549a00d admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:45.438975 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34be649d-8458-46df-a72d-c08a2549a00d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:45.440907 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34be649d-8458-46df-a72d-c08a2549a00d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:45.471474 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 969/1951] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:42:45 2022] POST /identity/v3/users => generated 292 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:42:45.477952 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5757f58a-5f61-4f2a-ba7b-2c0edb864eed admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:45.486616 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5757f58a-5f61-4f2a-ba7b-2c0edb864eed admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:45.491321 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08afd529-36ff-4951-a8aa-d5b081d96736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:45.491922 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08afd529-36ff-4951-a8aa-d5b081d96736 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:45.492197 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08afd529-36ff-4951-a8aa-d5b081d96736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:45.492465 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08afd529-36ff-4951-a8aa-d5b081d96736 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:45.493092 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08afd529-36ff-4951-a8aa-d5b081d96736 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:45.495406 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08afd529-36ff-4951-a8aa-d5b081d96736 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:45.511820 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 983/1952] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:42:45.513255 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be649d-8458-46df-a72d-c08a2549a00d admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:45.524340 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be649d-8458-46df-a72d-c08a2549a00d admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:45.529105 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190bd9e-9e33-4bb9-909e-d23b1de914a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:45.529843 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190bd9e-9e33-4bb9-909e-d23b1de914a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:45.530146 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190bd9e-9e33-4bb9-909e-d23b1de914a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:45.530417 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190bd9e-9e33-4bb9-909e-d23b1de914a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:45.531084 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c190bd9e-9e33-4bb9-909e-d23b1de914a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:45.532694 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c190bd9e-9e33-4bb9-909e-d23b1de914a1 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:45.544265 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 970/1953] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:42:45.549032 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08afd529-36ff-4951-a8aa-d5b081d96736 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:45.561113 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08afd529-36ff-4951-a8aa-d5b081d96736 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:45.566045 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5961b45-abef-49c8-88ed-2062285dfc7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:45.566848 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5961b45-abef-49c8-88ed-2062285dfc7e admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:45.567134 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5961b45-abef-49c8-88ed-2062285dfc7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:45.567408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5961b45-abef-49c8-88ed-2062285dfc7e admin admin] PATH_INFO: `/v3/projects/dbcfc825d7534d7d9f839f8c1a2e0560/users/3fa14bc0fa184dd69aea3a6ff082ce2e/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:45.585082 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5961b45-abef-49c8-88ed-2062285dfc7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbcfc825d7534d7d9f839f8c1a2e0560, user_id=3fa14bc0fa184dd69aea3a6ff082ce2e, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:45.588476 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5961b45-abef-49c8-88ed-2062285dfc7e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:45.602619 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 984/1954] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:45 2022] PUT /identity/v3/projects/dbcfc825d7534d7d9f839f8c1a2e0560/users/3fa14bc0fa184dd69aea3a6ff082ce2e/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:45.613668 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7dfd19-dca0-4f64-9298-4652585490be None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:45.614035 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7dfd19-dca0-4f64-9298-4652585490be None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:45.614307 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7dfd19-dca0-4f64-9298-4652585490be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:45.642652 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-5b7dfd19-dca0-4f64-9298-4652585490be None None] MFA Rules not processed for user `3fa14bc0fa184dd69aea3a6ff082ce2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:45.643858 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-5b7dfd19-dca0-4f64-9298-4652585490be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:45.710332 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 971/1955] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:42:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:42:45.714816 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5961b45-abef-49c8-88ed-2062285dfc7e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:45.741350 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:45.777246 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5961b45-abef-49c8-88ed-2062285dfc7e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:45.782025 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5d1c8a-bd4c-44a5-9427-516a724c9fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:45.783148 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d1c8a-bd4c-44a5-9427-516a724c9fc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:45.783148 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d1c8a-bd4c-44a5-9427-516a724c9fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:45.783314 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d1c8a-bd4c-44a5-9427-516a724c9fc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:45.783866 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5d1c8a-bd4c-44a5-9427-516a724c9fc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:45.785746 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5d1c8a-bd4c-44a5-9427-516a724c9fc6 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:45.821811 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 985/1956] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:42:45 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:45.822967 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7dfd19-dca0-4f64-9298-4652585490be None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:45.834057 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7dfd19-dca0-4f64-9298-4652585490be None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:45.892272 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b59235-e4ec-44de-ad5a-d87cf95fe3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:45.892874 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b59235-e4ec-44de-ad5a-d87cf95fe3a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:45.893148 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b59235-e4ec-44de-ad5a-d87cf95fe3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:45.893415 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b59235-e4ec-44de-ad5a-d87cf95fe3a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:45.894075 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b59235-e4ec-44de-ad5a-d87cf95fe3a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:45.895961 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b59235-e4ec-44de-ad5a-d87cf95fe3a1 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:45.917273 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 972/1957] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:42:45 2022] POST /identity/v3/users => generated 299 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:42:45.921235 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5d1c8a-bd4c-44a5-9427-516a724c9fc6 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:45.932012 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5d1c8a-bd4c-44a5-9427-516a724c9fc6 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:45.936470 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5046352-4ed9-49dc-88cc-c0cc463397b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:45.937086 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5046352-4ed9-49dc-88cc-c0cc463397b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:45.937365 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5046352-4ed9-49dc-88cc-c0cc463397b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:45.937630 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5046352-4ed9-49dc-88cc-c0cc463397b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:45.938297 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5046352-4ed9-49dc-88cc-c0cc463397b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:45.939871 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5046352-4ed9-49dc-88cc-c0cc463397b9 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:45.950150 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 986/1958] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:45.954010 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b59235-e4ec-44de-ad5a-d87cf95fe3a1 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:45.964682 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b59235-e4ec-44de-ad5a-d87cf95fe3a1 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:45.969246 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f43d839-1060-4a6a-bbae-5c0c678c59f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:45.969924 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f43d839-1060-4a6a-bbae-5c0c678c59f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:45.970205 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f43d839-1060-4a6a-bbae-5c0c678c59f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:45.970475 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f43d839-1060-4a6a-bbae-5c0c678c59f4 admin admin] PATH_INFO: `/v3/projects/9586e830bb164a3bad7f7713ea57081b/users/0976da7cc6c9406ca256ab798ce44cd6/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:45.979712 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:45.986784 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f43d839-1060-4a6a-bbae-5c0c678c59f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9586e830bb164a3bad7f7713ea57081b, user_id=0976da7cc6c9406ca256ab798ce44cd6, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:45.989589 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f43d839-1060-4a6a-bbae-5c0c678c59f4 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:46.004624 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 973/1959] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:45 2022] PUT /identity/v3/projects/9586e830bb164a3bad7f7713ea57081b/users/0976da7cc6c9406ca256ab798ce44cd6/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:46.009417 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5046352-4ed9-49dc-88cc-c0cc463397b9 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:46.019973 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5046352-4ed9-49dc-88cc-c0cc463397b9 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:46.083456 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec7a838-74af-46ba-ab0f-28544fcfe183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:46.084068 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec7a838-74af-46ba-ab0f-28544fcfe183 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:46.084349 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec7a838-74af-46ba-ab0f-28544fcfe183 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:46.084623 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec7a838-74af-46ba-ab0f-28544fcfe183 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:46.085253 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec7a838-74af-46ba-ab0f-28544fcfe183 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:46.087157 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec7a838-74af-46ba-ab0f-28544fcfe183 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:46.097598 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 987/1960] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:46 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:46.102149 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f43d839-1060-4a6a-bbae-5c0c678c59f4 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:46.112761 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f43d839-1060-4a6a-bbae-5c0c678c59f4 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:46.118598 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b0ebe-76fb-471e-90bc-51fc4fb35109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:46.119759 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b0ebe-76fb-471e-90bc-51fc4fb35109 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:46.120030 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b0ebe-76fb-471e-90bc-51fc4fb35109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:46.120293 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b0ebe-76fb-471e-90bc-51fc4fb35109 admin admin] PATH_INFO: `/v3/projects/9586e830bb164a3bad7f7713ea57081b/users/0976da7cc6c9406ca256ab798ce44cd6/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:46.122211 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82b0ebe-76fb-471e-90bc-51fc4fb35109 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9586e830bb164a3bad7f7713ea57081b, user_id=0976da7cc6c9406ca256ab798ce44cd6, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:46.125687 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82b0ebe-76fb-471e-90bc-51fc4fb35109 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:46.136800 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 974/1961] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:46 2022] PUT /identity/v3/projects/9586e830bb164a3bad7f7713ea57081b/users/0976da7cc6c9406ca256ab798ce44cd6/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:46.147118 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238a88a2-1425-4aff-b44c-46038fcb4986 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:46.147429 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238a88a2-1425-4aff-b44c-46038fcb4986 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:46.147716 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238a88a2-1425-4aff-b44c-46038fcb4986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:46.174550 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-238a88a2-1425-4aff-b44c-46038fcb4986 None None] MFA Rules not processed for user `0976da7cc6c9406ca256ab798ce44cd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:46.175726 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-238a88a2-1425-4aff-b44c-46038fcb4986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:46.245450 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 988/1962] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:42:46 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:46.256172 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b0ebe-76fb-471e-90bc-51fc4fb35109 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:46.257832 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c82b0ebe-76fb-471e-90bc-51fc4fb35109 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:46.336432 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b0ebe-76fb-471e-90bc-51fc4fb35109 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:46.340279 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67537ef1-89aa-4c2d-a321-d3831dab80d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:46.341037 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67537ef1-89aa-4c2d-a321-d3831dab80d9 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:46.341320 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67537ef1-89aa-4c2d-a321-d3831dab80d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:46.341590 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67537ef1-89aa-4c2d-a321-d3831dab80d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:46.406489 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67537ef1-89aa-4c2d-a321-d3831dab80d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:46.408461 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67537ef1-89aa-4c2d-a321-d3831dab80d9 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:46.419337 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 975/1963] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:42:46.457054 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238a88a2-1425-4aff-b44c-46038fcb4986 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:46.458488 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-238a88a2-1425-4aff-b44c-46038fcb4986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:46.481686 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:46.531777 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238a88a2-1425-4aff-b44c-46038fcb4986 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:46.536873 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a90375-1a94-4dc2-bacd-25674395f46e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:46.537670 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a90375-1a94-4dc2-bacd-25674395f46e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:46.537963 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a90375-1a94-4dc2-bacd-25674395f46e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:46.538055 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a90375-1a94-4dc2-bacd-25674395f46e admin admin] PATH_INFO: `/v3/users/a5055d2bc3ce498a8a47863ce51f9975` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:46.539400 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a90375-1a94-4dc2-bacd-25674395f46e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5055d2bc3ce498a8a47863ce51f9975)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:46.543371 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a90375-1a94-4dc2-bacd-25674395f46e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:46.598008 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e1a90375-1a94-4dc2-bacd-25674395f46e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5055d2bc3ce498a8a47863ce51f9975'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:46.598697 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e1a90375-1a94-4dc2-bacd-25674395f46e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5055d2bc3ce498a8a47863ce51f9975'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:46.613331 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e1a90375-1a94-4dc2-bacd-25674395f46e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5055d2bc3ce498a8a47863ce51f9975'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:46.621659 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e1a90375-1a94-4dc2-bacd-25674395f46e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5055d2bc3ce498a8a47863ce51f9975'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:46.634354 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e1a90375-1a94-4dc2-bacd-25674395f46e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5055d2bc3ce498a8a47863ce51f9975'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:46.643183 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 989/1964] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:42:46 2022] DELETE /identity/v3/users/a5055d2bc3ce498a8a47863ce51f9975 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:46.647673 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef06199-4769-4d38-ae32-88cb38f2b611 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:46.648138 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef06199-4769-4d38-ae32-88cb38f2b611 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:46.648487 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef06199-4769-4d38-ae32-88cb38f2b611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:46.669177 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-4ef06199-4769-4d38-ae32-88cb38f2b611 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:46.670321 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4ef06199-4769-4d38-ae32-88cb38f2b611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:46.734612 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 976/1965] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:42:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:46.744890 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a90375-1a94-4dc2-bacd-25674395f46e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:46.746366 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e1a90375-1a94-4dc2-bacd-25674395f46e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:46.814399 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a90375-1a94-4dc2-bacd-25674395f46e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:46.820444 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68048011-fbf1-4e72-8d9b-bacb14f6125a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:46.821060 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68048011-fbf1-4e72-8d9b-bacb14f6125a service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:46.822125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68048011-fbf1-4e72-8d9b-bacb14f6125a service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:46.822454 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68048011-fbf1-4e72-8d9b-bacb14f6125a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:46.832014 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68048011-fbf1-4e72-8d9b-bacb14f6125a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:46.833446 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68048011-fbf1-4e72-8d9b-bacb14f6125a service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:46.844808 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 990/1966] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:42:47.118535 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef06199-4769-4d38-ae32-88cb38f2b611 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:47.120181 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4ef06199-4769-4d38-ae32-88cb38f2b611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:47.135245 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef06199-4769-4d38-ae32-88cb38f2b611 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:47.140435 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89ee807-e1fc-4579-9558-b0c8e830b920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:47.141142 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ee807-e1fc-4579-9558-b0c8e830b920 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:47.141338 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ee807-e1fc-4579-9558-b0c8e830b920 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:47.141596 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ee807-e1fc-4579-9558-b0c8e830b920 admin admin] PATH_INFO: `/v3/projects/f825cc3d4dfa417f8fd8f8a10ec2c248` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:47.142754 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89ee807-e1fc-4579-9558-b0c8e830b920 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f825cc3d4dfa417f8fd8f8a10ec2c248)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:47.144727 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89ee807-e1fc-4579-9558-b0c8e830b920 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:47.170697 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:47.194021 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c89ee807-e1fc-4579-9558-b0c8e830b920 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f825cc3d4dfa417f8fd8f8a10ec2c248', 'request_id': 'req-c89ee807-e1fc-4579-9558-b0c8e830b920'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:47.202052 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c89ee807-e1fc-4579-9558-b0c8e830b920 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f825cc3d4dfa417f8fd8f8a10ec2c248', 'request_id': 'req-c89ee807-e1fc-4579-9558-b0c8e830b920'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:47.208181 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c89ee807-e1fc-4579-9558-b0c8e830b920 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f825cc3d4dfa417f8fd8f8a10ec2c248', 'request_id': 'req-c89ee807-e1fc-4579-9558-b0c8e830b920'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:47.220129 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c89ee807-e1fc-4579-9558-b0c8e830b920 admin admin] The token cache is being invalidate because project f825cc3d4dfa417f8fd8f8a10ec2c248 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:42:47.220432 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c89ee807-e1fc-4579-9558-b0c8e830b920 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:47.221063 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c89ee807-e1fc-4579-9558-b0c8e830b920 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:47.222591 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 977/1967] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:42:47 2022] DELETE /identity/v3/projects/f825cc3d4dfa417f8fd8f8a10ec2c248 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:47.226485 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68048011-fbf1-4e72-8d9b-bacb14f6125a service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:47.235947 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68048011-fbf1-4e72-8d9b-bacb14f6125a service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:47.277942 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:47.296078 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6ce983-74ef-4c39-9ccc-32c6a9fc16ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:47.297006 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ce983-74ef-4c39-9ccc-32c6a9fc16ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:47.297341 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ce983-74ef-4c39-9ccc-32c6a9fc16ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:47.298209 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ce983-74ef-4c39-9ccc-32c6a9fc16ca admin admin] PATH_INFO: `/v3/users/817e2da961a84fe0921cac5d3e84943f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:47.298927 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6ce983-74ef-4c39-9ccc-32c6a9fc16ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=817e2da961a84fe0921cac5d3e84943f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:47.300948 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6ce983-74ef-4c39-9ccc-32c6a9fc16ca admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:47.346059 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2c6ce983-74ef-4c39-9ccc-32c6a9fc16ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '817e2da961a84fe0921cac5d3e84943f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:47.346634 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2c6ce983-74ef-4c39-9ccc-32c6a9fc16ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '817e2da961a84fe0921cac5d3e84943f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:47.358286 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2c6ce983-74ef-4c39-9ccc-32c6a9fc16ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '817e2da961a84fe0921cac5d3e84943f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:47.365436 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2c6ce983-74ef-4c39-9ccc-32c6a9fc16ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '817e2da961a84fe0921cac5d3e84943f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:47.372286 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2c6ce983-74ef-4c39-9ccc-32c6a9fc16ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '817e2da961a84fe0921cac5d3e84943f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:47.378264 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 991/1968] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:42:47 2022] DELETE /identity/v3/users/817e2da961a84fe0921cac5d3e84943f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:47.643411 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89ee807-e1fc-4579-9558-b0c8e830b920 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:47.645044 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c89ee807-e1fc-4579-9558-b0c8e830b920 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:47.665064 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89ee807-e1fc-4579-9558-b0c8e830b920 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:47.723836 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0661ce-aab7-495d-a8af-f2397c2cbfb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:47.724474 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0661ce-aab7-495d-a8af-f2397c2cbfb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:47.724751 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0661ce-aab7-495d-a8af-f2397c2cbfb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:47.725019 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0661ce-aab7-495d-a8af-f2397c2cbfb8 admin admin] PATH_INFO: `/v3/projects/18c7cd1babcf409094c60f6af3973bc7` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:47.726235 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0661ce-aab7-495d-a8af-f2397c2cbfb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18c7cd1babcf409094c60f6af3973bc7)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:47.728206 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0661ce-aab7-495d-a8af-f2397c2cbfb8 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:47.778671 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-cd0661ce-aab7-495d-a8af-f2397c2cbfb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18c7cd1babcf409094c60f6af3973bc7', 'request_id': 'req-cd0661ce-aab7-495d-a8af-f2397c2cbfb8'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:47.788368 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-cd0661ce-aab7-495d-a8af-f2397c2cbfb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18c7cd1babcf409094c60f6af3973bc7', 'request_id': 'req-cd0661ce-aab7-495d-a8af-f2397c2cbfb8'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:47.794965 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-cd0661ce-aab7-495d-a8af-f2397c2cbfb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18c7cd1babcf409094c60f6af3973bc7', 'request_id': 'req-cd0661ce-aab7-495d-a8af-f2397c2cbfb8'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:47.805814 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-cd0661ce-aab7-495d-a8af-f2397c2cbfb8 admin admin] The token cache is being invalidate because project 18c7cd1babcf409094c60f6af3973bc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:42:47.806136 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-cd0661ce-aab7-495d-a8af-f2397c2cbfb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:47.807287 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-cd0661ce-aab7-495d-a8af-f2397c2cbfb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:47.808758 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 978/1969] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:42:47 2022] DELETE /identity/v3/projects/18c7cd1babcf409094c60f6af3973bc7 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:47.817256 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6960e34-1284-41e9-b08b-e0d546e457aa None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:47.817545 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6960e34-1284-41e9-b08b-e0d546e457aa None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:47.817898 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6960e34-1284-41e9-b08b-e0d546e457aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:47.839968 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-a6960e34-1284-41e9-b08b-e0d546e457aa None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:47.841161 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a6960e34-1284-41e9-b08b-e0d546e457aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:47.904876 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 992/1970] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:42:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:47.912726 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd4926-f20b-406f-95ff-78318ee8a1a4 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:47.913009 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd4926-f20b-406f-95ff-78318ee8a1a4 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:47.913284 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cd4926-f20b-406f-95ff-78318ee8a1a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:47.934314 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-22cd4926-f20b-406f-95ff-78318ee8a1a4 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:47.935504 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-22cd4926-f20b-406f-95ff-78318ee8a1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:47.943561 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 979/1971] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:42:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:47.947619 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6960e34-1284-41e9-b08b-e0d546e457aa None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:47.957109 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6960e34-1284-41e9-b08b-e0d546e457aa None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:47.961632 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d5ff01-cd07-4574-9e56-e0ecb7be7954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:47.962268 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5ff01-cd07-4574-9e56-e0ecb7be7954 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:47.962601 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5ff01-cd07-4574-9e56-e0ecb7be7954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:47.962872 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5ff01-cd07-4574-9e56-e0ecb7be7954 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:47.963506 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d5ff01-cd07-4574-9e56-e0ecb7be7954 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:42:47.963805 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d5ff01-cd07-4574-9e56-e0ecb7be7954 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:47.965395 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d5ff01-cd07-4574-9e56-e0ecb7be7954 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:47.977917 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 993/1972] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:42:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:42:47.982935 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cd4926-f20b-406f-95ff-78318ee8a1a4 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:47.994513 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cd4926-f20b-406f-95ff-78318ee8a1a4 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:47.999974 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07de1a7e-2a1e-4f9c-bed8-0fb8c93d92f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:48.000579 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de1a7e-2a1e-4f9c-bed8-0fb8c93d92f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:48.000861 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de1a7e-2a1e-4f9c-bed8-0fb8c93d92f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:48.001173 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07de1a7e-2a1e-4f9c-bed8-0fb8c93d92f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:48.001804 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07de1a7e-2a1e-4f9c-bed8-0fb8c93d92f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:48.003762 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07de1a7e-2a1e-4f9c-bed8-0fb8c93d92f7 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:48.037062 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 980/1973] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:42:47 2022] POST /identity/v3/projects => generated 409 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:48.041587 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d5ff01-cd07-4574-9e56-e0ecb7be7954 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:48.053367 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d5ff01-cd07-4574-9e56-e0ecb7be7954 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:48.110099 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:48.127437 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c82405-9b6e-4ed4-85a8-c8b51676a0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:48.128378 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c82405-9b6e-4ed4-85a8-c8b51676a0c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:48.128787 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c82405-9b6e-4ed4-85a8-c8b51676a0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:48.129066 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c82405-9b6e-4ed4-85a8-c8b51676a0c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:48.129708 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c82405-9b6e-4ed4-85a8-c8b51676a0c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:48.131667 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c82405-9b6e-4ed4-85a8-c8b51676a0c4 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:48.158835 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 994/1974] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:42:48 2022] POST /identity/v3/users => generated 305 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:48.160052 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07de1a7e-2a1e-4f9c-bed8-0fb8c93d92f7 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:48.188422 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07de1a7e-2a1e-4f9c-bed8-0fb8c93d92f7 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:48.212185 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614dd7e5-427e-45ea-b578-f5edc64cde75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:48.214416 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614dd7e5-427e-45ea-b578-f5edc64cde75 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:48.217883 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614dd7e5-427e-45ea-b578-f5edc64cde75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:48.217883 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614dd7e5-427e-45ea-b578-f5edc64cde75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:48.217883 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614dd7e5-427e-45ea-b578-f5edc64cde75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:48.221131 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614dd7e5-427e-45ea-b578-f5edc64cde75 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:48.248245 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 981/1975] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:48 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:48.252356 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c82405-9b6e-4ed4-85a8-c8b51676a0c4 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:48.271949 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c82405-9b6e-4ed4-85a8-c8b51676a0c4 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:48.282339 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ca92cd-558f-490a-bf2f-65bf8f14e012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:48.282971 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca92cd-558f-490a-bf2f-65bf8f14e012 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:48.283253 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca92cd-558f-490a-bf2f-65bf8f14e012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:48.283527 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca92cd-558f-490a-bf2f-65bf8f14e012 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:48.284402 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ca92cd-558f-490a-bf2f-65bf8f14e012 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:48.286697 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ca92cd-558f-490a-bf2f-65bf8f14e012 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:48.318344 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 995/1976] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:48 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:48.319582 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614dd7e5-427e-45ea-b578-f5edc64cde75 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:48.325313 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:48.330885 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614dd7e5-427e-45ea-b578-f5edc64cde75 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:48.338258 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3073f1e-7750-4e35-93ef-726f626cc6a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:48.339193 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3073f1e-7750-4e35-93ef-726f626cc6a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:48.339900 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3073f1e-7750-4e35-93ef-726f626cc6a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:48.340408 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3073f1e-7750-4e35-93ef-726f626cc6a0 admin admin] PATH_INFO: `/v3/projects/822aea8ec9d546d59049b9dda9f21b73/users/37f646f6d06a4cadb19d1bc9b116c19e/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:48.361334 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3073f1e-7750-4e35-93ef-726f626cc6a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=822aea8ec9d546d59049b9dda9f21b73, user_id=37f646f6d06a4cadb19d1bc9b116c19e, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:48.364412 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3073f1e-7750-4e35-93ef-726f626cc6a0 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:48.383882 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 982/1977] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:48 2022] PUT /identity/v3/projects/822aea8ec9d546d59049b9dda9f21b73/users/37f646f6d06a4cadb19d1bc9b116c19e/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:48.395794 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650e80e-e039-4844-837e-d977b37fca6c None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:48.396084 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650e80e-e039-4844-837e-d977b37fca6c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:48.396393 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e650e80e-e039-4844-837e-d977b37fca6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:48.427645 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-e650e80e-e039-4844-837e-d977b37fca6c None None] MFA Rules not processed for user `37f646f6d06a4cadb19d1bc9b116c19e`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:48.428769 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e650e80e-e039-4844-837e-d977b37fca6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:48.495209 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 996/1978] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:42:48 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:48.499984 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3073f1e-7750-4e35-93ef-726f626cc6a0 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:48.568996 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3073f1e-7750-4e35-93ef-726f626cc6a0 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:48.574384 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ca411e-9df6-46fc-99d6-5f856a2a0628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:48.575000 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca411e-9df6-46fc-99d6-5f856a2a0628 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:48.575279 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca411e-9df6-46fc-99d6-5f856a2a0628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:48.575925 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ca411e-9df6-46fc-99d6-5f856a2a0628 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:48.576417 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ca411e-9df6-46fc-99d6-5f856a2a0628 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:48.578489 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ca411e-9df6-46fc-99d6-5f856a2a0628 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:48.606670 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 983/1979] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:42:48 2022] POST /identity/v3/projects => generated 407 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:42:48.611237 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e650e80e-e039-4844-837e-d977b37fca6c None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:48.622092 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e650e80e-e039-4844-837e-d977b37fca6c None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:48.708972 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e637b0f-df03-4632-a5a6-b1b9132edd07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:48.709673 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e637b0f-df03-4632-a5a6-b1b9132edd07 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:48.709984 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e637b0f-df03-4632-a5a6-b1b9132edd07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:48.710321 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e637b0f-df03-4632-a5a6-b1b9132edd07 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:48.711321 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e637b0f-df03-4632-a5a6-b1b9132edd07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:48.714985 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e637b0f-df03-4632-a5a6-b1b9132edd07 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:48.738834 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 997/1980] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:42:48 2022] POST /identity/v3/users => generated 310 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:42:48.743303 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ca411e-9df6-46fc-99d6-5f856a2a0628 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:48.756665 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ca411e-9df6-46fc-99d6-5f856a2a0628 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:48.763127 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bb403f-d87d-4163-851e-27fc588deade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:48.763921 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb403f-d87d-4163-851e-27fc588deade admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:48.764240 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb403f-d87d-4163-851e-27fc588deade admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:48.764991 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb403f-d87d-4163-851e-27fc588deade admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:48.765206 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bb403f-d87d-4163-851e-27fc588deade admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:48.766972 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bb403f-d87d-4163-851e-27fc588deade admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:48.786701 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 984/1981] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:48 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:48.791582 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e637b0f-df03-4632-a5a6-b1b9132edd07 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:48.804861 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e637b0f-df03-4632-a5a6-b1b9132edd07 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:48.810960 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e458a00e-9f5e-406e-9555-26abf92ed84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:48.812931 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e458a00e-9f5e-406e-9555-26abf92ed84d admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:48.813479 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e458a00e-9f5e-406e-9555-26abf92ed84d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:48.814115 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e458a00e-9f5e-406e-9555-26abf92ed84d admin admin] PATH_INFO: `/v3/projects/d728706ab37246508ec0966905a0d5e9/users/c56ccdc6a61148058f4656d3b6e6fa68/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:48.840956 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e458a00e-9f5e-406e-9555-26abf92ed84d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d728706ab37246508ec0966905a0d5e9, user_id=c56ccdc6a61148058f4656d3b6e6fa68, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:48.845712 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e458a00e-9f5e-406e-9555-26abf92ed84d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:48.861069 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 998/1982] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:48 2022] PUT /identity/v3/projects/d728706ab37246508ec0966905a0d5e9/users/c56ccdc6a61148058f4656d3b6e6fa68/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:42:48.865047 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bb403f-d87d-4163-851e-27fc588deade admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:48.875288 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bb403f-d87d-4163-851e-27fc588deade admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:48.933048 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87969df6-5cc9-4fbd-ba22-857d4d03f753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:48.933652 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87969df6-5cc9-4fbd-ba22-857d4d03f753 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:48.933966 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87969df6-5cc9-4fbd-ba22-857d4d03f753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:48.934241 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87969df6-5cc9-4fbd-ba22-857d4d03f753 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:48.934882 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87969df6-5cc9-4fbd-ba22-857d4d03f753 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:48.936462 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87969df6-5cc9-4fbd-ba22-857d4d03f753 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:48.946310 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 985/1983] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:42:48 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:42:48.950815 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e458a00e-9f5e-406e-9555-26abf92ed84d admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:48.962126 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e458a00e-9f5e-406e-9555-26abf92ed84d admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:48.967843 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6dda23-ac2c-444f-a5aa-106c1dbb81e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:48.968546 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6dda23-ac2c-444f-a5aa-106c1dbb81e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:48.968832 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6dda23-ac2c-444f-a5aa-106c1dbb81e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:48.969111 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6dda23-ac2c-444f-a5aa-106c1dbb81e5 admin admin] PATH_INFO: `/v3/projects/d728706ab37246508ec0966905a0d5e9/users/c56ccdc6a61148058f4656d3b6e6fa68/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:48.971072 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6dda23-ac2c-444f-a5aa-106c1dbb81e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d728706ab37246508ec0966905a0d5e9, user_id=c56ccdc6a61148058f4656d3b6e6fa68, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:48.974348 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6dda23-ac2c-444f-a5aa-106c1dbb81e5 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:48.985334 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 999/1984] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:42:48 2022] PUT /identity/v3/projects/d728706ab37246508ec0966905a0d5e9/users/c56ccdc6a61148058f4656d3b6e6fa68/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:48.996437 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252aa38a-b995-4ca3-9f15-ad15f8079cb0 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:48.996719 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252aa38a-b995-4ca3-9f15-ad15f8079cb0 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:48.997296 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252aa38a-b995-4ca3-9f15-ad15f8079cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:49.028394 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-252aa38a-b995-4ca3-9f15-ad15f8079cb0 None None] MFA Rules not processed for user `c56ccdc6a61148058f4656d3b6e6fa68`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:49.029535 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-252aa38a-b995-4ca3-9f15-ad15f8079cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:49.090419 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:49.130990 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 986/1985] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:42:48 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:49.142385 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6dda23-ac2c-444f-a5aa-106c1dbb81e5 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:49.144149 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8b6dda23-ac2c-444f-a5aa-106c1dbb81e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:49.156341 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:49.204541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6dda23-ac2c-444f-a5aa-106c1dbb81e5 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:49.208084 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868e5e59-ecb2-4fe1-ad27-9df088ab372b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:49.208690 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e5e59-ecb2-4fe1-ad27-9df088ab372b service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:49.208972 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e5e59-ecb2-4fe1-ad27-9df088ab372b service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:49.209236 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e5e59-ecb2-4fe1-ad27-9df088ab372b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:49.218443 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868e5e59-ecb2-4fe1-ad27-9df088ab372b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:49.219797 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868e5e59-ecb2-4fe1-ad27-9df088ab372b service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:49.229416 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1000/1986] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:49 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:42:49.458222 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252aa38a-b995-4ca3-9f15-ad15f8079cb0 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:49.468563 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252aa38a-b995-4ca3-9f15-ad15f8079cb0 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:49.472697 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fb0883-2906-4a6d-8c87-6461e79943f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:49.473474 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fb0883-2906-4a6d-8c87-6461e79943f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:49.473878 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fb0883-2906-4a6d-8c87-6461e79943f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:49.474238 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fb0883-2906-4a6d-8c87-6461e79943f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:49.550174 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fb0883-2906-4a6d-8c87-6461e79943f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:49.551884 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fb0883-2906-4a6d-8c87-6461e79943f1 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:49.563063 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 987/1987] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:49 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:42:51.059088 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868e5e59-ecb2-4fe1-ad27-9df088ab372b service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:51.067737 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868e5e59-ecb2-4fe1-ad27-9df088ab372b service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:51.071455 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e57f0c3-fbb1-41e6-89ce-6472560112ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:51.073249 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e57f0c3-fbb1-41e6-89ce-6472560112ed service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:51.073535 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e57f0c3-fbb1-41e6-89ce-6472560112ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:51.073810 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e57f0c3-fbb1-41e6-89ce-6472560112ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:51.075635 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9e57f0c3-fbb1-41e6-89ce-6472560112ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:51.141128 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e57f0c3-fbb1-41e6-89ce-6472560112ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:51.142664 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e57f0c3-fbb1-41e6-89ce-6472560112ed service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:51.152990 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1001/1988] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:51 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:42:58.104448 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fb0883-2906-4a6d-8c87-6461e79943f1 service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:58.236549 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fb0883-2906-4a6d-8c87-6461e79943f1 service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:58.243627 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfc3cb7-5b86-4fe9-bf1a-506dfde03e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:58.245212 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc3cb7-5b86-4fe9-bf1a-506dfde03e32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:58.245682 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc3cb7-5b86-4fe9-bf1a-506dfde03e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:58.246445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc3cb7-5b86-4fe9-bf1a-506dfde03e32 admin admin] PATH_INFO: `/v3/users/37f646f6d06a4cadb19d1bc9b116c19e` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:58.247804 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfc3cb7-5b86-4fe9-bf1a-506dfde03e32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37f646f6d06a4cadb19d1bc9b116c19e)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:58.251283 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfc3cb7-5b86-4fe9-bf1a-506dfde03e32 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:58.336194 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1dfc3cb7-5b86-4fe9-bf1a-506dfde03e32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37f646f6d06a4cadb19d1bc9b116c19e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:58.352076 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1dfc3cb7-5b86-4fe9-bf1a-506dfde03e32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37f646f6d06a4cadb19d1bc9b116c19e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:58.353757 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1dfc3cb7-5b86-4fe9-bf1a-506dfde03e32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37f646f6d06a4cadb19d1bc9b116c19e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:58.369666 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1dfc3cb7-5b86-4fe9-bf1a-506dfde03e32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37f646f6d06a4cadb19d1bc9b116c19e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:58.370643 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1dfc3cb7-5b86-4fe9-bf1a-506dfde03e32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37f646f6d06a4cadb19d1bc9b116c19e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:58.391508 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:58.410847 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 988/1989] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:42:58 2022] DELETE /identity/v3/users/37f646f6d06a4cadb19d1bc9b116c19e => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:58.415965 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c1414-865d-4ad3-9b93-02a52581157b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:58.416494 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c1414-865d-4ad3-9b93-02a52581157b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:58.416760 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c1414-865d-4ad3-9b93-02a52581157b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:58.460335 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d57c1414-865d-4ad3-9b93-02a52581157b None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:58.461891 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d57c1414-865d-4ad3-9b93-02a52581157b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:58.539517 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1002/1990] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:42:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:58.547452 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfc3cb7-5b86-4fe9-bf1a-506dfde03e32 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:58.549401 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-1dfc3cb7-5b86-4fe9-bf1a-506dfde03e32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:58.610540 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfc3cb7-5b86-4fe9-bf1a-506dfde03e32 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:58.614124 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1612283-cdb2-4551-8cb0-b2b55a8cbed0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:58.614725 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1612283-cdb2-4551-8cb0-b2b55a8cbed0 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:58.615001 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1612283-cdb2-4551-8cb0-b2b55a8cbed0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:58.615270 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1612283-cdb2-4551-8cb0-b2b55a8cbed0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:58.624863 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1612283-cdb2-4551-8cb0-b2b55a8cbed0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:58.626327 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1612283-cdb2-4551-8cb0-b2b55a8cbed0 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:58.635671 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 989/1991] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:42:58.913820 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57c1414-865d-4ad3-9b93-02a52581157b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:58.916480 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d57c1414-865d-4ad3-9b93-02a52581157b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:58.933954 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57c1414-865d-4ad3-9b93-02a52581157b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:58.940656 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3151da4-202e-461a-b61b-22d1977ff16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:58.941309 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3151da4-202e-461a-b61b-22d1977ff16b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:58.941589 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3151da4-202e-461a-b61b-22d1977ff16b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:58.941949 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3151da4-202e-461a-b61b-22d1977ff16b admin admin] PATH_INFO: `/v3/projects/822aea8ec9d546d59049b9dda9f21b73` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:58.943102 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3151da4-202e-461a-b61b-22d1977ff16b admin admin] RBAC: Authorizing `identity:delete_project(project_id=822aea8ec9d546d59049b9dda9f21b73)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:58.945205 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3151da4-202e-461a-b61b-22d1977ff16b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:58.996930 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:59.007106 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a3151da4-202e-461a-b61b-22d1977ff16b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '822aea8ec9d546d59049b9dda9f21b73', 'request_id': 'req-a3151da4-202e-461a-b61b-22d1977ff16b'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:59.013709 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a3151da4-202e-461a-b61b-22d1977ff16b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '822aea8ec9d546d59049b9dda9f21b73', 'request_id': 'req-a3151da4-202e-461a-b61b-22d1977ff16b'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:59.022733 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a3151da4-202e-461a-b61b-22d1977ff16b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '822aea8ec9d546d59049b9dda9f21b73', 'request_id': 'req-a3151da4-202e-461a-b61b-22d1977ff16b'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:59.037234 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a3151da4-202e-461a-b61b-22d1977ff16b admin admin] The token cache is being invalidate because project 822aea8ec9d546d59049b9dda9f21b73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:42:59.037547 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a3151da4-202e-461a-b61b-22d1977ff16b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:59.038547 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a3151da4-202e-461a-b61b-22d1977ff16b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:59.047393 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1003/1992] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:42:58 2022] DELETE /identity/v3/projects/822aea8ec9d546d59049b9dda9f21b73 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:59.052000 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1612283-cdb2-4551-8cb0-b2b55a8cbed0 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:59.066083 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1612283-cdb2-4551-8cb0-b2b55a8cbed0 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:59.128726 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37ecc85-393b-4266-a55b-9f911d74707f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:59.130099 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ecc85-393b-4266-a55b-9f911d74707f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:59.130378 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ecc85-393b-4266-a55b-9f911d74707f admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:59.130649 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ecc85-393b-4266-a55b-9f911d74707f admin admin] PATH_INFO: `/v3/users/c56ccdc6a61148058f4656d3b6e6fa68` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:59.131794 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37ecc85-393b-4266-a55b-9f911d74707f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c56ccdc6a61148058f4656d3b6e6fa68)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:59.134276 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37ecc85-393b-4266-a55b-9f911d74707f admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:59.187140 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c37ecc85-393b-4266-a55b-9f911d74707f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c56ccdc6a61148058f4656d3b6e6fa68'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:59.197889 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c37ecc85-393b-4266-a55b-9f911d74707f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c56ccdc6a61148058f4656d3b6e6fa68'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:59.199475 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c37ecc85-393b-4266-a55b-9f911d74707f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c56ccdc6a61148058f4656d3b6e6fa68'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:59.209276 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c37ecc85-393b-4266-a55b-9f911d74707f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c56ccdc6a61148058f4656d3b6e6fa68'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:59.210102 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c37ecc85-393b-4266-a55b-9f911d74707f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c56ccdc6a61148058f4656d3b6e6fa68'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:59.239683 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 990/1993] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:42:59 2022] DELETE /identity/v3/users/c56ccdc6a61148058f4656d3b6e6fa68 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:59.434568 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3151da4-202e-461a-b61b-22d1977ff16b admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:59.436374 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a3151da4-202e-461a-b61b-22d1977ff16b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:59.451556 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3151da4-202e-461a-b61b-22d1977ff16b admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:59.522182 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b84417-162a-4fba-b3e2-b126d0e96621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:59.522876 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b84417-162a-4fba-b3e2-b126d0e96621 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:59.523183 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b84417-162a-4fba-b3e2-b126d0e96621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:59.523533 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b84417-162a-4fba-b3e2-b126d0e96621 admin admin] PATH_INFO: `/v3/users/3fa14bc0fa184dd69aea3a6ff082ce2e` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:59.524832 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b84417-162a-4fba-b3e2-b126d0e96621 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fa14bc0fa184dd69aea3a6ff082ce2e)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:59.527056 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b84417-162a-4fba-b3e2-b126d0e96621 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:59.546695 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37ecc85-393b-4266-a55b-9f911d74707f admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:59.547831 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c37ecc85-393b-4266-a55b-9f911d74707f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:59.564520 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37ecc85-393b-4266-a55b-9f911d74707f admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:59.572476 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d6ab92-6cf8-4439-a666-627da8210266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:59.573326 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d6ab92-6cf8-4439-a666-627da8210266 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:59.573668 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d6ab92-6cf8-4439-a666-627da8210266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:59.574080 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d6ab92-6cf8-4439-a666-627da8210266 admin admin] PATH_INFO: `/v3/projects/d728706ab37246508ec0966905a0d5e9` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:59.575322 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d6ab92-6cf8-4439-a666-627da8210266 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d728706ab37246508ec0966905a0d5e9)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:59.577586 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d6ab92-6cf8-4439-a666-627da8210266 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:59.587878 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:59.619692 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e2b84417-162a-4fba-b3e2-b126d0e96621 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fa14bc0fa184dd69aea3a6ff082ce2e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:59.620362 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e2b84417-162a-4fba-b3e2-b126d0e96621 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fa14bc0fa184dd69aea3a6ff082ce2e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:59.641417 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e2b84417-162a-4fba-b3e2-b126d0e96621 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fa14bc0fa184dd69aea3a6ff082ce2e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:59.652514 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e2b84417-162a-4fba-b3e2-b126d0e96621 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fa14bc0fa184dd69aea3a6ff082ce2e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:59.662052 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-11d6ab92-6cf8-4439-a666-627da8210266 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd728706ab37246508ec0966905a0d5e9', 'request_id': 'req-11d6ab92-6cf8-4439-a666-627da8210266'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:59.667590 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e2b84417-162a-4fba-b3e2-b126d0e96621 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fa14bc0fa184dd69aea3a6ff082ce2e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:59.673614 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-11d6ab92-6cf8-4439-a666-627da8210266 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd728706ab37246508ec0966905a0d5e9', 'request_id': 'req-11d6ab92-6cf8-4439-a666-627da8210266'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:59.678983 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1004/1994] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:42:59 2022] DELETE /identity/v3/users/3fa14bc0fa184dd69aea3a6ff082ce2e => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:59.683379 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca9408-1ed0-426c-8d7c-51d86eef881f None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:59.683717 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca9408-1ed0-426c-8d7c-51d86eef881f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:59.684046 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca9408-1ed0-426c-8d7c-51d86eef881f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:59.692211 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-11d6ab92-6cf8-4439-a666-627da8210266 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd728706ab37246508ec0966905a0d5e9', 'request_id': 'req-11d6ab92-6cf8-4439-a666-627da8210266'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:59.713281 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-59ca9408-1ed0-426c-8d7c-51d86eef881f None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:59.713391 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-59ca9408-1ed0-426c-8d7c-51d86eef881f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:59.723305 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-11d6ab92-6cf8-4439-a666-627da8210266 admin admin] The token cache is being invalidate because project d728706ab37246508ec0966905a0d5e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:42:59.724437 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-11d6ab92-6cf8-4439-a666-627da8210266 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:59.725253 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-11d6ab92-6cf8-4439-a666-627da8210266 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:42:59.729115 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 991/1995] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:42:59 2022] DELETE /identity/v3/projects/d728706ab37246508ec0966905a0d5e9 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:42:59.735087 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df109e1-2352-46b3-a96c-b3321f953d7f None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:59.735482 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df109e1-2352-46b3-a96c-b3321f953d7f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:59.735855 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df109e1-2352-46b3-a96c-b3321f953d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:59.758591 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-2df109e1-2352-46b3-a96c-b3321f953d7f None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:59.760509 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-2df109e1-2352-46b3-a96c-b3321f953d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:59.769742 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:42:59.787461 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1005/1996] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:42:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:59.797176 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca9408-1ed0-426c-8d7c-51d86eef881f None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:59.798600 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-59ca9408-1ed0-426c-8d7c-51d86eef881f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:59.822904 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 992/1997] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:42:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:42:59.830642 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435dc2e-ee41-4a8d-9e3e-851459924781 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:59.830997 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435dc2e-ee41-4a8d-9e3e-851459924781 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:59.831349 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435dc2e-ee41-4a8d-9e3e-851459924781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:59.851242 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-2435dc2e-ee41-4a8d-9e3e-851459924781 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:42:59.853036 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-2435dc2e-ee41-4a8d-9e3e-851459924781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:42:59.859784 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca9408-1ed0-426c-8d7c-51d86eef881f None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:59.861223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 993/1998] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:42:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:42:59.863366 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ef2853-c281-4ea1-aa42-c0ea5c616ea3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:59.863974 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef2853-c281-4ea1-aa42-c0ea5c616ea3 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:59.864314 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef2853-c281-4ea1-aa42-c0ea5c616ea3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:59.864537 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef2853-c281-4ea1-aa42-c0ea5c616ea3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:59.865336 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2435dc2e-ee41-4a8d-9e3e-851459924781 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:59.874865 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ef2853-c281-4ea1-aa42-c0ea5c616ea3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:59.875287 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2435dc2e-ee41-4a8d-9e3e-851459924781 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:59.876305 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ef2853-c281-4ea1-aa42-c0ea5c616ea3 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:59.879966 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e701c5-0639-4871-bce8-932d3c4bbba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:59.880557 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e701c5-0639-4871-bce8-932d3c4bbba9 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:59.880830 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e701c5-0639-4871-bce8-932d3c4bbba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:59.881119 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e701c5-0639-4871-bce8-932d3c4bbba9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:59.881798 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e701c5-0639-4871-bce8-932d3c4bbba9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:42:59.882125 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e701c5-0639-4871-bce8-932d3c4bbba9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:59.883781 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e701c5-0639-4871-bce8-932d3c4bbba9 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:59.886143 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1006/1999] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:42:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:42:59.899202 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 994/2000] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:42:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:42:59.900318 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ef2853-c281-4ea1-aa42-c0ea5c616ea3 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:59.909880 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ef2853-c281-4ea1-aa42-c0ea5c616ea3 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:42:59.914607 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21c9752-5e34-49bb-b487-cb431d6b1d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:42:59.915624 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c9752-5e34-49bb-b487-cb431d6b1d48 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:42:59.915898 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c9752-5e34-49bb-b487-cb431d6b1d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:42:59.916165 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c9752-5e34-49bb-b487-cb431d6b1d48 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:42:59.916788 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21c9752-5e34-49bb-b487-cb431d6b1d48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:42:59.918734 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21c9752-5e34-49bb-b487-cb431d6b1d48 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:42:59.946395 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1007/2001] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:42:59 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:42:59.950841 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e701c5-0639-4871-bce8-932d3c4bbba9 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:42:59.961130 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e701c5-0639-4871-bce8-932d3c4bbba9 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:00.019816 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97737c92-8f97-4467-adcb-9fc4768006ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:00.020528 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97737c92-8f97-4467-adcb-9fc4768006ab admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:00.020803 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97737c92-8f97-4467-adcb-9fc4768006ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:00.021066 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97737c92-8f97-4467-adcb-9fc4768006ab admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:00.021698 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97737c92-8f97-4467-adcb-9fc4768006ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:00.023651 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97737c92-8f97-4467-adcb-9fc4768006ab admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:00.048121 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 995/2002] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:42:59 2022] POST /identity/v3/users => generated 291 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:00.053706 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21c9752-5e34-49bb-b487-cb431d6b1d48 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:00.065049 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21c9752-5e34-49bb-b487-cb431d6b1d48 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:00.069948 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ff9652-e37a-49a9-a9e2-3a95f4d537dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:00.070554 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff9652-e37a-49a9-a9e2-3a95f4d537dd admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:00.070856 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff9652-e37a-49a9-a9e2-3a95f4d537dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:00.071158 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff9652-e37a-49a9-a9e2-3a95f4d537dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:00.071808 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ff9652-e37a-49a9-a9e2-3a95f4d537dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:00.073372 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ff9652-e37a-49a9-a9e2-3a95f4d537dd admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:00.083659 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1008/2003] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:43:00.087498 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97737c92-8f97-4467-adcb-9fc4768006ab admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:00.097552 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97737c92-8f97-4467-adcb-9fc4768006ab admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:00.102165 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fef5b83-5e41-40c7-8dc6-0356d61c110d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:00.103278 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef5b83-5e41-40c7-8dc6-0356d61c110d admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:00.103551 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef5b83-5e41-40c7-8dc6-0356d61c110d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:00.103817 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef5b83-5e41-40c7-8dc6-0356d61c110d admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:00.104451 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fef5b83-5e41-40c7-8dc6-0356d61c110d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:00.106064 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fef5b83-5e41-40c7-8dc6-0356d61c110d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:00.107356 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:00.120109 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 996/2004] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:00.123869 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ff9652-e37a-49a9-a9e2-3a95f4d537dd admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:00.134823 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ff9652-e37a-49a9-a9e2-3a95f4d537dd admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:00.139927 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eb5b38-3375-4e2f-90bc-89cbfbbfd1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:00.140716 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb5b38-3375-4e2f-90bc-89cbfbbfd1d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:00.141387 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb5b38-3375-4e2f-90bc-89cbfbbfd1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:00.141387 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eb5b38-3375-4e2f-90bc-89cbfbbfd1d5 admin admin] PATH_INFO: `/v3/projects/e2cebcfe611a4042904e7ef539658a84/users/535f18a274294fce8b39ebc8f7fb651b/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:00.159710 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eb5b38-3375-4e2f-90bc-89cbfbbfd1d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2cebcfe611a4042904e7ef539658a84, user_id=535f18a274294fce8b39ebc8f7fb651b, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:00.163805 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eb5b38-3375-4e2f-90bc-89cbfbbfd1d5 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:00.168243 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fef5b83-5e41-40c7-8dc6-0356d61c110d admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:00.170905 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-0fef5b83-5e41-40c7-8dc6-0356d61c110d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:00.184766 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fef5b83-5e41-40c7-8dc6-0356d61c110d admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:00.188005 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1009/2005] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:00 2022] PUT /identity/v3/projects/e2cebcfe611a4042904e7ef539658a84/users/535f18a274294fce8b39ebc8f7fb651b/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:00.206966 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d2b6d-fe8e-49b0-b6df-e310128e2c8d None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:00.207514 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d2b6d-fe8e-49b0-b6df-e310128e2c8d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:00.208148 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d2b6d-fe8e-49b0-b6df-e310128e2c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:00.239168 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-e43d2b6d-fe8e-49b0-b6df-e310128e2c8d None None] MFA Rules not processed for user `535f18a274294fce8b39ebc8f7fb651b`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:00.240380 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e43d2b6d-fe8e-49b0-b6df-e310128e2c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:00.246642 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a6eddd-defd-4321-9cb7-00f0e3b47cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:00.247286 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6eddd-defd-4321-9cb7-00f0e3b47cf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:00.247564 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6eddd-defd-4321-9cb7-00f0e3b47cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:00.247836 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6eddd-defd-4321-9cb7-00f0e3b47cf1 admin admin] PATH_INFO: `/v3/projects/dbcfc825d7534d7d9f839f8c1a2e0560` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:00.248964 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a6eddd-defd-4321-9cb7-00f0e3b47cf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbcfc825d7534d7d9f839f8c1a2e0560)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:00.251178 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a6eddd-defd-4321-9cb7-00f0e3b47cf1 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:00.316212 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-20a6eddd-defd-4321-9cb7-00f0e3b47cf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbcfc825d7534d7d9f839f8c1a2e0560', 'request_id': 'req-20a6eddd-defd-4321-9cb7-00f0e3b47cf1'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:00.330338 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-20a6eddd-defd-4321-9cb7-00f0e3b47cf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbcfc825d7534d7d9f839f8c1a2e0560', 'request_id': 'req-20a6eddd-defd-4321-9cb7-00f0e3b47cf1'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:00.331041 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1010/2006] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:43:00 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:43:00.335157 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d2b6d-fe8e-49b0-b6df-e310128e2c8d None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:00.340971 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-20a6eddd-defd-4321-9cb7-00f0e3b47cf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbcfc825d7534d7d9f839f8c1a2e0560', 'request_id': 'req-20a6eddd-defd-4321-9cb7-00f0e3b47cf1'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:00.346065 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d2b6d-fe8e-49b0-b6df-e310128e2c8d None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:00.354047 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-20a6eddd-defd-4321-9cb7-00f0e3b47cf1 admin admin] The token cache is being invalidate because project dbcfc825d7534d7d9f839f8c1a2e0560 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:43:00.354412 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-20a6eddd-defd-4321-9cb7-00f0e3b47cf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:00.355153 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-20a6eddd-defd-4321-9cb7-00f0e3b47cf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:00.359950 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 997/2007] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:43:00 2022] DELETE /identity/v3/projects/dbcfc825d7534d7d9f839f8c1a2e0560 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:00.361328 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a6eddd-defd-4321-9cb7-00f0e3b47cf1 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:00.362466 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-20a6eddd-defd-4321-9cb7-00f0e3b47cf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:00.388148 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:00.441575 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bd3858-ae74-4c21-8103-3002c0ce8726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:00.442461 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd3858-ae74-4c21-8103-3002c0ce8726 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:00.442886 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd3858-ae74-4c21-8103-3002c0ce8726 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:00.443239 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd3858-ae74-4c21-8103-3002c0ce8726 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:00.443971 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bd3858-ae74-4c21-8103-3002c0ce8726 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:00.445087 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a6eddd-defd-4321-9cb7-00f0e3b47cf1 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:00.451056 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bd3858-ae74-4c21-8103-3002c0ce8726 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:00.451304 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f93447-5ba8-49b8-a417-8ff190000a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:00.451882 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f93447-5ba8-49b8-a417-8ff190000a6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:00.452157 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f93447-5ba8-49b8-a417-8ff190000a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:00.452482 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f93447-5ba8-49b8-a417-8ff190000a6e admin admin] PATH_INFO: `/v3/users/0976da7cc6c9406ca256ab798ce44cd6` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:00.454200 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f93447-5ba8-49b8-a417-8ff190000a6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0976da7cc6c9406ca256ab798ce44cd6)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:00.457346 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f93447-5ba8-49b8-a417-8ff190000a6e admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:00.497369 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:00.506953 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1011/2008] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:43:00 2022] POST /identity/v3/projects => generated 379 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:43:00.511654 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bd3858-ae74-4c21-8103-3002c0ce8726 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:00.515862 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d2bd3858-ae74-4c21-8103-3002c0ce8726 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:00.523171 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-61f93447-5ba8-49b8-a417-8ff190000a6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0976da7cc6c9406ca256ab798ce44cd6'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:00.532821 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-61f93447-5ba8-49b8-a417-8ff190000a6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0976da7cc6c9406ca256ab798ce44cd6'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:00.534279 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-61f93447-5ba8-49b8-a417-8ff190000a6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0976da7cc6c9406ca256ab798ce44cd6'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:00.545724 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-61f93447-5ba8-49b8-a417-8ff190000a6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0976da7cc6c9406ca256ab798ce44cd6'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:00.547707 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-61f93447-5ba8-49b8-a417-8ff190000a6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0976da7cc6c9406ca256ab798ce44cd6'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:00.581910 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 998/2009] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:43:00 2022] DELETE /identity/v3/users/0976da7cc6c9406ca256ab798ce44cd6 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:00.608404 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bd3858-ae74-4c21-8103-3002c0ce8726 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:00.682519 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32787952-10fe-451e-80db-7e8e8eb494dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:00.683176 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32787952-10fe-451e-80db-7e8e8eb494dd admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:00.683413 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32787952-10fe-451e-80db-7e8e8eb494dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:00.683687 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32787952-10fe-451e-80db-7e8e8eb494dd admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:00.684306 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32787952-10fe-451e-80db-7e8e8eb494dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:00.686261 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32787952-10fe-451e-80db-7e8e8eb494dd admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:00.710868 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1012/2010] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:43:00 2022] POST /identity/v3/users => generated 296 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:00.714728 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f93447-5ba8-49b8-a417-8ff190000a6e admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:00.716677 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-61f93447-5ba8-49b8-a417-8ff190000a6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:00.731965 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f93447-5ba8-49b8-a417-8ff190000a6e admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:00.738231 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834d2b6-66e3-401b-8600-b0e311427ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:00.738935 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834d2b6-66e3-401b-8600-b0e311427ac2 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:00.739216 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834d2b6-66e3-401b-8600-b0e311427ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:00.739521 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834d2b6-66e3-401b-8600-b0e311427ac2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:00.741905 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2834d2b6-66e3-401b-8600-b0e311427ac2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:00.741905 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2834d2b6-66e3-401b-8600-b0e311427ac2 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:00.761390 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 999/2011] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:00.765390 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32787952-10fe-451e-80db-7e8e8eb494dd admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:00.783124 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32787952-10fe-451e-80db-7e8e8eb494dd admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:00.800069 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8f88b3-91ad-4e1b-bae3-af1e464556e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:00.800802 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f88b3-91ad-4e1b-bae3-af1e464556e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:00.801077 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f88b3-91ad-4e1b-bae3-af1e464556e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:00.801346 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f88b3-91ad-4e1b-bae3-af1e464556e7 admin admin] PATH_INFO: `/v3/projects/31bb57c9afec4002a58549dee76af5c8/users/003eeb123fd84cad9fdca9c0e459ed2c/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:00.840653 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8f88b3-91ad-4e1b-bae3-af1e464556e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31bb57c9afec4002a58549dee76af5c8, user_id=003eeb123fd84cad9fdca9c0e459ed2c, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:00.845206 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8f88b3-91ad-4e1b-bae3-af1e464556e7 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:00.882595 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1013/2012] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:00 2022] PUT /identity/v3/projects/31bb57c9afec4002a58549dee76af5c8/users/003eeb123fd84cad9fdca9c0e459ed2c/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:00.886699 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834d2b6-66e3-401b-8600-b0e311427ac2 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:00.898676 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834d2b6-66e3-401b-8600-b0e311427ac2 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:00.967364 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8f88b3-91ad-4e1b-bae3-af1e464556e7 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:00.969893 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7f8f88b3-91ad-4e1b-bae3-af1e464556e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:00.999390 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7065d3b-3fcb-4bf0-a701-8bda6b4d65b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:01.000557 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7065d3b-3fcb-4bf0-a701-8bda6b4d65b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:01.000557 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7065d3b-3fcb-4bf0-a701-8bda6b4d65b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:01.001402 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7065d3b-3fcb-4bf0-a701-8bda6b4d65b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:01.002084 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7065d3b-3fcb-4bf0-a701-8bda6b4d65b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:01.003188 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:01.005775 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7065d3b-3fcb-4bf0-a701-8bda6b4d65b3 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:01.025444 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1000/2013] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:01.037757 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7065d3b-3fcb-4bf0-a701-8bda6b4d65b3 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:01.054444 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7065d3b-3fcb-4bf0-a701-8bda6b4d65b3 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:01.060571 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2cc11d-4406-4143-939d-d18ebeb1c438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:01.061245 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cc11d-4406-4143-939d-d18ebeb1c438 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:01.061521 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cc11d-4406-4143-939d-d18ebeb1c438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:01.062024 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2cc11d-4406-4143-939d-d18ebeb1c438 admin admin] PATH_INFO: `/v3/projects/31bb57c9afec4002a58549dee76af5c8/users/003eeb123fd84cad9fdca9c0e459ed2c/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:01.064358 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2cc11d-4406-4143-939d-d18ebeb1c438 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31bb57c9afec4002a58549dee76af5c8, user_id=003eeb123fd84cad9fdca9c0e459ed2c, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:01.067679 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2cc11d-4406-4143-939d-d18ebeb1c438 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:01.085926 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1001/2014] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:01 2022] PUT /identity/v3/projects/31bb57c9afec4002a58549dee76af5c8/users/003eeb123fd84cad9fdca9c0e459ed2c/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:01.088262 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8f88b3-91ad-4e1b-bae3-af1e464556e7 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:01.092855 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc7ecff-ed6e-4c64-b0e6-3341187a0a15 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:01.093134 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc7ecff-ed6e-4c64-b0e6-3341187a0a15 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:01.093603 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc7ecff-ed6e-4c64-b0e6-3341187a0a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:01.127035 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-ebc7ecff-ed6e-4c64-b0e6-3341187a0a15 None None] MFA Rules not processed for user `003eeb123fd84cad9fdca9c0e459ed2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:01.128320 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ebc7ecff-ed6e-4c64-b0e6-3341187a0a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:01.166308 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b30fbcc-58fa-4a7a-a6e7-eb73258cf08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:01.167924 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30fbcc-58fa-4a7a-a6e7-eb73258cf08c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:01.168316 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30fbcc-58fa-4a7a-a6e7-eb73258cf08c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:01.168694 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30fbcc-58fa-4a7a-a6e7-eb73258cf08c admin admin] PATH_INFO: `/v3/projects/9586e830bb164a3bad7f7713ea57081b` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:01.170638 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b30fbcc-58fa-4a7a-a6e7-eb73258cf08c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9586e830bb164a3bad7f7713ea57081b)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:01.172590 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b30fbcc-58fa-4a7a-a6e7-eb73258cf08c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:01.212077 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1002/2015] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:43:01 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:43:01.222929 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc7ecff-ed6e-4c64-b0e6-3341187a0a15 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:01.224370 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ebc7ecff-ed6e-4c64-b0e6-3341187a0a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:01.227822 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3b30fbcc-58fa-4a7a-a6e7-eb73258cf08c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9586e830bb164a3bad7f7713ea57081b', 'request_id': 'req-3b30fbcc-58fa-4a7a-a6e7-eb73258cf08c'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:01.231928 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:01.235177 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3b30fbcc-58fa-4a7a-a6e7-eb73258cf08c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9586e830bb164a3bad7f7713ea57081b', 'request_id': 'req-3b30fbcc-58fa-4a7a-a6e7-eb73258cf08c'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:01.246779 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3b30fbcc-58fa-4a7a-a6e7-eb73258cf08c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9586e830bb164a3bad7f7713ea57081b', 'request_id': 'req-3b30fbcc-58fa-4a7a-a6e7-eb73258cf08c'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:01.258223 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3b30fbcc-58fa-4a7a-a6e7-eb73258cf08c admin admin] The token cache is being invalidate because project 9586e830bb164a3bad7f7713ea57081b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:43:01.258584 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3b30fbcc-58fa-4a7a-a6e7-eb73258cf08c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:01.260139 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3b30fbcc-58fa-4a7a-a6e7-eb73258cf08c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:01.267445 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1014/2016] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:43:00 2022] DELETE /identity/v3/projects/9586e830bb164a3bad7f7713ea57081b => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:01.268823 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b966d0b-3b31-47b4-842c-2aab150dc08b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:01.269102 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b966d0b-3b31-47b4-842c-2aab150dc08b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:01.269371 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b966d0b-3b31-47b4-842c-2aab150dc08b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:01.290195 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-4b966d0b-3b31-47b4-842c-2aab150dc08b None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:01.291715 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4b966d0b-3b31-47b4-842c-2aab150dc08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:01.302738 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc7ecff-ed6e-4c64-b0e6-3341187a0a15 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:01.360647 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c06e222-9b9c-4aff-a0ed-ddd43a457982 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:01.361297 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06e222-9b9c-4aff-a0ed-ddd43a457982 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:01.361577 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06e222-9b9c-4aff-a0ed-ddd43a457982 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:01.361869 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06e222-9b9c-4aff-a0ed-ddd43a457982 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:01.362120 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:01.363876 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-6c06e222-9b9c-4aff-a0ed-ddd43a457982 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:01.372411 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1015/2017] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:43:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:43:01.381137 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2f143-9917-49d4-a1d8-fc2aa2fa2f6d None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:01.381451 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2f143-9917-49d4-a1d8-fc2aa2fa2f6d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:01.381675 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2f143-9917-49d4-a1d8-fc2aa2fa2f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:01.403014 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-faf2f143-9917-49d4-a1d8-fc2aa2fa2f6d None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:01.404216 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-faf2f143-9917-49d4-a1d8-fc2aa2fa2f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:01.412349 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1016/2018] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:43:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:43:01.416998 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf2f143-9917-49d4-a1d8-fc2aa2fa2f6d None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:01.427449 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf2f143-9917-49d4-a1d8-fc2aa2fa2f6d None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:01.432682 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460fae78-6481-47e2-9274-6bcae3774c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:01.433104 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c06e222-9b9c-4aff-a0ed-ddd43a457982 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:01.433304 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460fae78-6481-47e2-9274-6bcae3774c0e admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:01.433565 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460fae78-6481-47e2-9274-6bcae3774c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:01.433837 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460fae78-6481-47e2-9274-6bcae3774c0e admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:01.434499 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460fae78-6481-47e2-9274-6bcae3774c0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:43:01.434624 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c06e222-9b9c-4aff-a0ed-ddd43a457982 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:01.434803 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460fae78-6481-47e2-9274-6bcae3774c0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:01.436391 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460fae78-6481-47e2-9274-6bcae3774c0e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:01.444353 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1003/2019] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:01 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:43:01.451204 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1017/2020] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:43:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:43:01.452557 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c06e222-9b9c-4aff-a0ed-ddd43a457982 service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:01.463015 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c06e222-9b9c-4aff-a0ed-ddd43a457982 service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:01.468016 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9508ec-0e0d-49d9-b158-bda3bf1aa342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:01.468726 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9508ec-0e0d-49d9-b158-bda3bf1aa342 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:01.469058 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9508ec-0e0d-49d9-b158-bda3bf1aa342 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:01.469521 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9508ec-0e0d-49d9-b158-bda3bf1aa342 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:01.470602 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9508ec-0e0d-49d9-b158-bda3bf1aa342 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:01.472901 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9508ec-0e0d-49d9-b158-bda3bf1aa342 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:01.502287 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1004/2021] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:43:01 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:43:01.506019 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460fae78-6481-47e2-9274-6bcae3774c0e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:01.516257 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460fae78-6481-47e2-9274-6bcae3774c0e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:01.576339 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb54c0cd-d040-4084-9d9b-1cf88080b9f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:01.576947 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb54c0cd-d040-4084-9d9b-1cf88080b9f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:01.577225 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb54c0cd-d040-4084-9d9b-1cf88080b9f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:01.577495 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb54c0cd-d040-4084-9d9b-1cf88080b9f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:01.578160 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb54c0cd-d040-4084-9d9b-1cf88080b9f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:01.580092 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb54c0cd-d040-4084-9d9b-1cf88080b9f7 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:01.609713 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1018/2022] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:43:01 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:01.614463 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9508ec-0e0d-49d9-b158-bda3bf1aa342 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:01.626764 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9508ec-0e0d-49d9-b158-bda3bf1aa342 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:01.632288 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f2b3c4-bdb5-4608-89e6-9b8e0f34a928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:01.633018 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2b3c4-bdb5-4608-89e6-9b8e0f34a928 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:01.633274 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2b3c4-bdb5-4608-89e6-9b8e0f34a928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:01.633574 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2b3c4-bdb5-4608-89e6-9b8e0f34a928 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:01.634267 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f2b3c4-bdb5-4608-89e6-9b8e0f34a928 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:01.635914 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f2b3c4-bdb5-4608-89e6-9b8e0f34a928 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:01.647928 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1005/2023] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:01.655192 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb54c0cd-d040-4084-9d9b-1cf88080b9f7 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:01.668978 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb54c0cd-d040-4084-9d9b-1cf88080b9f7 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:01.673994 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18b112-c212-49a3-8181-36dc17c82102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:01.674793 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18b112-c212-49a3-8181-36dc17c82102 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:01.679323 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18b112-c212-49a3-8181-36dc17c82102 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:01.679323 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18b112-c212-49a3-8181-36dc17c82102 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:01.679323 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c18b112-c212-49a3-8181-36dc17c82102 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:01.679323 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c18b112-c212-49a3-8181-36dc17c82102 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:01.693899 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1019/2024] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:01.697576 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f2b3c4-bdb5-4608-89e6-9b8e0f34a928 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:01.708270 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f2b3c4-bdb5-4608-89e6-9b8e0f34a928 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:01.714761 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1773d2c-06d3-4495-8cae-c5275595275c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:01.715361 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1773d2c-06d3-4495-8cae-c5275595275c admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:01.715637 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1773d2c-06d3-4495-8cae-c5275595275c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:01.715908 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1773d2c-06d3-4495-8cae-c5275595275c admin admin] PATH_INFO: `/v3/projects/07612fca609845c4ae38b958c53e0e69/users/5cbcfa011bb342aaaaca09b8060d0b6a/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:01.732560 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1773d2c-06d3-4495-8cae-c5275595275c admin admin] RBAC: Authorizing `identity:create_grant(project_id=07612fca609845c4ae38b958c53e0e69, user_id=5cbcfa011bb342aaaaca09b8060d0b6a, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:01.735248 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1773d2c-06d3-4495-8cae-c5275595275c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:01.751365 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1006/2025] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:01 2022] PUT /identity/v3/projects/07612fca609845c4ae38b958c53e0e69/users/5cbcfa011bb342aaaaca09b8060d0b6a/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:01.762794 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfde708-4b3f-4b65-8773-9b915cf339cc None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:01.763136 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfde708-4b3f-4b65-8773-9b915cf339cc None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:01.763413 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfde708-4b3f-4b65-8773-9b915cf339cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:01.795203 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-cbfde708-4b3f-4b65-8773-9b915cf339cc None None] MFA Rules not processed for user `5cbcfa011bb342aaaaca09b8060d0b6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:01.796401 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-cbfde708-4b3f-4b65-8773-9b915cf339cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:01.865404 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1020/2026] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:43:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:43:01.880691 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1773d2c-06d3-4495-8cae-c5275595275c admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:01.882425 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c1773d2c-06d3-4495-8cae-c5275595275c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:01.902486 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:01.909216 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfde708-4b3f-4b65-8773-9b915cf339cc None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:01.910734 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-cbfde708-4b3f-4b65-8773-9b915cf339cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:01.954058 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1773d2c-06d3-4495-8cae-c5275595275c admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:01.958157 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363019c8-e5c0-4ca5-b0ff-2baaf54063d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:01.958835 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363019c8-e5c0-4ca5-b0ff-2baaf54063d9 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:01.959177 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363019c8-e5c0-4ca5-b0ff-2baaf54063d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:01.959513 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363019c8-e5c0-4ca5-b0ff-2baaf54063d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:01.971184 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363019c8-e5c0-4ca5-b0ff-2baaf54063d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:01.972373 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363019c8-e5c0-4ca5-b0ff-2baaf54063d9 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:01.975512 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:01.985179 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1007/2027] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:01 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:43:01.999744 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfde708-4b3f-4b65-8773-9b915cf339cc None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:02.004955 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d7dbd5-6026-4781-a2c9-aa2d0dd5e898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:02.005760 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d7dbd5-6026-4781-a2c9-aa2d0dd5e898 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:02.006224 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d7dbd5-6026-4781-a2c9-aa2d0dd5e898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:02.006811 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d7dbd5-6026-4781-a2c9-aa2d0dd5e898 admin admin] PATH_INFO: `/v3/users/535f18a274294fce8b39ebc8f7fb651b` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:02.008204 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d7dbd5-6026-4781-a2c9-aa2d0dd5e898 admin admin] RBAC: Authorizing `identity:delete_user(user_id=535f18a274294fce8b39ebc8f7fb651b)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:02.010526 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d7dbd5-6026-4781-a2c9-aa2d0dd5e898 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:02.073317 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-43d7dbd5-6026-4781-a2c9-aa2d0dd5e898 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '535f18a274294fce8b39ebc8f7fb651b'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:02.074066 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-43d7dbd5-6026-4781-a2c9-aa2d0dd5e898 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '535f18a274294fce8b39ebc8f7fb651b'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:02.088914 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-43d7dbd5-6026-4781-a2c9-aa2d0dd5e898 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '535f18a274294fce8b39ebc8f7fb651b'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:02.098064 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-43d7dbd5-6026-4781-a2c9-aa2d0dd5e898 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '535f18a274294fce8b39ebc8f7fb651b'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:02.107721 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-43d7dbd5-6026-4781-a2c9-aa2d0dd5e898 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '535f18a274294fce8b39ebc8f7fb651b'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:02.115050 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1021/2028] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:43:01 2022] DELETE /identity/v3/users/535f18a274294fce8b39ebc8f7fb651b => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:02.119614 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f68e41-3a92-4daf-a54b-bd68b7c962b3 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:02.120247 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f68e41-3a92-4daf-a54b-bd68b7c962b3 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:02.120247 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f68e41-3a92-4daf-a54b-bd68b7c962b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:02.147128 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-84f68e41-3a92-4daf-a54b-bd68b7c962b3 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:02.148805 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-84f68e41-3a92-4daf-a54b-bd68b7c962b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:02.222422 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1008/2029] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:43:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:43:02.232082 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d7dbd5-6026-4781-a2c9-aa2d0dd5e898 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:02.233963 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-43d7dbd5-6026-4781-a2c9-aa2d0dd5e898 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:02.310225 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d7dbd5-6026-4781-a2c9-aa2d0dd5e898 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:02.316438 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ff3b6e-b311-4457-a79f-1c54ea71c9e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:02.317075 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff3b6e-b311-4457-a79f-1c54ea71c9e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:02.317439 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff3b6e-b311-4457-a79f-1c54ea71c9e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:02.317619 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff3b6e-b311-4457-a79f-1c54ea71c9e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:02.331614 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ff3b6e-b311-4457-a79f-1c54ea71c9e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:02.333260 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ff3b6e-b311-4457-a79f-1c54ea71c9e9 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:02.350055 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1022/2030] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:43:02.642893 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f68e41-3a92-4daf-a54b-bd68b7c962b3 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:02.644580 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-84f68e41-3a92-4daf-a54b-bd68b7c962b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:02.657360 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f68e41-3a92-4daf-a54b-bd68b7c962b3 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:02.662057 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41446f70-b59c-4e09-a331-120a86b59d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:02.662790 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41446f70-b59c-4e09-a331-120a86b59d54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:02.663161 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41446f70-b59c-4e09-a331-120a86b59d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:02.663552 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41446f70-b59c-4e09-a331-120a86b59d54 admin admin] PATH_INFO: `/v3/projects/e2cebcfe611a4042904e7ef539658a84` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:02.664818 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41446f70-b59c-4e09-a331-120a86b59d54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2cebcfe611a4042904e7ef539658a84)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:02.666852 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41446f70-b59c-4e09-a331-120a86b59d54 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:02.720492 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-41446f70-b59c-4e09-a331-120a86b59d54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2cebcfe611a4042904e7ef539658a84', 'request_id': 'req-41446f70-b59c-4e09-a331-120a86b59d54'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:02.728177 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-41446f70-b59c-4e09-a331-120a86b59d54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2cebcfe611a4042904e7ef539658a84', 'request_id': 'req-41446f70-b59c-4e09-a331-120a86b59d54'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:02.736071 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-41446f70-b59c-4e09-a331-120a86b59d54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2cebcfe611a4042904e7ef539658a84', 'request_id': 'req-41446f70-b59c-4e09-a331-120a86b59d54'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:02.747630 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-41446f70-b59c-4e09-a331-120a86b59d54 admin admin] The token cache is being invalidate because project e2cebcfe611a4042904e7ef539658a84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:43:02.748406 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-41446f70-b59c-4e09-a331-120a86b59d54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:02.749327 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-41446f70-b59c-4e09-a331-120a86b59d54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:02.750845 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1009/2031] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:43:02 2022] DELETE /identity/v3/projects/e2cebcfe611a4042904e7ef539658a84 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:02.754799 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ff3b6e-b311-4457-a79f-1c54ea71c9e9 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:02.765036 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ff3b6e-b311-4457-a79f-1c54ea71c9e9 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:02.829251 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bd564f-f779-4dfc-bd5d-a14ff2ec2402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:02.830016 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd564f-f779-4dfc-bd5d-a14ff2ec2402 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:02.830499 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd564f-f779-4dfc-bd5d-a14ff2ec2402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:02.830846 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd564f-f779-4dfc-bd5d-a14ff2ec2402 admin admin] PATH_INFO: `/v3/users/003eeb123fd84cad9fdca9c0e459ed2c` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:02.832050 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bd564f-f779-4dfc-bd5d-a14ff2ec2402 admin admin] RBAC: Authorizing `identity:delete_user(user_id=003eeb123fd84cad9fdca9c0e459ed2c)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:02.834618 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bd564f-f779-4dfc-bd5d-a14ff2ec2402 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:02.848994 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:02.890240 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d4bd564f-f779-4dfc-bd5d-a14ff2ec2402 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '003eeb123fd84cad9fdca9c0e459ed2c'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:02.891424 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d4bd564f-f779-4dfc-bd5d-a14ff2ec2402 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '003eeb123fd84cad9fdca9c0e459ed2c'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:02.904447 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d4bd564f-f779-4dfc-bd5d-a14ff2ec2402 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '003eeb123fd84cad9fdca9c0e459ed2c'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:02.912704 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d4bd564f-f779-4dfc-bd5d-a14ff2ec2402 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '003eeb123fd84cad9fdca9c0e459ed2c'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:02.920660 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d4bd564f-f779-4dfc-bd5d-a14ff2ec2402 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '003eeb123fd84cad9fdca9c0e459ed2c'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:02.926647 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1023/2032] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:43:02 2022] DELETE /identity/v3/users/003eeb123fd84cad9fdca9c0e459ed2c => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:03.218674 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41446f70-b59c-4e09-a331-120a86b59d54 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:03.220181 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-41446f70-b59c-4e09-a331-120a86b59d54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:03.244686 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41446f70-b59c-4e09-a331-120a86b59d54 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:03.312192 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:03.380211 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b4e8b4-1584-4455-b85f-e3f20f4c4af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:03.380876 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4e8b4-1584-4455-b85f-e3f20f4c4af2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:03.381149 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4e8b4-1584-4455-b85f-e3f20f4c4af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:03.381422 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4e8b4-1584-4455-b85f-e3f20f4c4af2 admin admin] PATH_INFO: `/v3/projects/31bb57c9afec4002a58549dee76af5c8` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:03.384369 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b4e8b4-1584-4455-b85f-e3f20f4c4af2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31bb57c9afec4002a58549dee76af5c8)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:03.394877 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b4e8b4-1584-4455-b85f-e3f20f4c4af2 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:03.461403 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d8b4e8b4-1584-4455-b85f-e3f20f4c4af2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31bb57c9afec4002a58549dee76af5c8', 'request_id': 'req-d8b4e8b4-1584-4455-b85f-e3f20f4c4af2'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:03.470281 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d8b4e8b4-1584-4455-b85f-e3f20f4c4af2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31bb57c9afec4002a58549dee76af5c8', 'request_id': 'req-d8b4e8b4-1584-4455-b85f-e3f20f4c4af2'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:03.484021 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d8b4e8b4-1584-4455-b85f-e3f20f4c4af2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31bb57c9afec4002a58549dee76af5c8', 'request_id': 'req-d8b4e8b4-1584-4455-b85f-e3f20f4c4af2'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:03.497476 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d8b4e8b4-1584-4455-b85f-e3f20f4c4af2 admin admin] The token cache is being invalidate because project 31bb57c9afec4002a58549dee76af5c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:43:03.497757 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d8b4e8b4-1584-4455-b85f-e3f20f4c4af2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:03.498457 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d8b4e8b4-1584-4455-b85f-e3f20f4c4af2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:03.500483 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1010/2033] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:43:03 2022] DELETE /identity/v3/projects/31bb57c9afec4002a58549dee76af5c8 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:03.505192 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9b6d5-649a-4981-a97d-69860d73d0cb None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:03.505512 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9b6d5-649a-4981-a97d-69860d73d0cb None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:03.505734 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9b6d5-649a-4981-a97d-69860d73d0cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:03.529960 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-29b9b6d5-649a-4981-a97d-69860d73d0cb None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:03.531258 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-29b9b6d5-649a-4981-a97d-69860d73d0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:03.608667 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1024/2034] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:43:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:03.616523 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d032de-f8b3-4625-9d50-470ba4a08d09 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:03.616813 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d032de-f8b3-4625-9d50-470ba4a08d09 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:03.617087 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d032de-f8b3-4625-9d50-470ba4a08d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:03.643719 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-76d032de-f8b3-4625-9d50-470ba4a08d09 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:03.645006 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-76d032de-f8b3-4625-9d50-470ba4a08d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:03.658449 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1011/2035] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:43:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:03.663077 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b9b6d5-649a-4981-a97d-69860d73d0cb None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:03.674266 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b9b6d5-649a-4981-a97d-69860d73d0cb None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:03.679187 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd7a3f-a530-459f-b07d-f32542f657ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:03.679902 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd7a3f-a530-459f-b07d-f32542f657ea admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:03.680308 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd7a3f-a530-459f-b07d-f32542f657ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:03.680890 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd7a3f-a530-459f-b07d-f32542f657ea admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:03.681457 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcd7a3f-a530-459f-b07d-f32542f657ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:43:03.681880 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcd7a3f-a530-459f-b07d-f32542f657ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:03.683567 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcd7a3f-a530-459f-b07d-f32542f657ea admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:03.696584 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1025/2036] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:43:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:43:03.701020 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d032de-f8b3-4625-9d50-470ba4a08d09 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:03.711046 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d032de-f8b3-4625-9d50-470ba4a08d09 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:03.717950 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e546e1-da72-46d5-95fb-e4e13d9ce2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:03.719209 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e546e1-da72-46d5-95fb-e4e13d9ce2e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:03.719700 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e546e1-da72-46d5-95fb-e4e13d9ce2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:03.720164 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e546e1-da72-46d5-95fb-e4e13d9ce2e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:03.720862 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e546e1-da72-46d5-95fb-e4e13d9ce2e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:03.724378 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e546e1-da72-46d5-95fb-e4e13d9ce2e2 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:03.757447 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1012/2037] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:43:03 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:03.761070 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd7a3f-a530-459f-b07d-f32542f657ea admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:03.771284 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd7a3f-a530-459f-b07d-f32542f657ea admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:03.857439 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb36a1f9-4061-47a2-a75f-359987400d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:03.858429 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36a1f9-4061-47a2-a75f-359987400d17 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:03.858937 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36a1f9-4061-47a2-a75f-359987400d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:03.859388 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36a1f9-4061-47a2-a75f-359987400d17 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:03.860227 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb36a1f9-4061-47a2-a75f-359987400d17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:03.866025 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb36a1f9-4061-47a2-a75f-359987400d17 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:03.907220 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1026/2038] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:43:03 2022] POST /identity/v3/users => generated 287 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:03.908644 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e546e1-da72-46d5-95fb-e4e13d9ce2e2 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:03.922875 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e546e1-da72-46d5-95fb-e4e13d9ce2e2 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:03.928625 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74cd218-5b36-4b8d-accc-067fab8a8efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:03.928625 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74cd218-5b36-4b8d-accc-067fab8a8efc admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:03.928886 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74cd218-5b36-4b8d-accc-067fab8a8efc admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:03.929128 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74cd218-5b36-4b8d-accc-067fab8a8efc admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:03.930715 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74cd218-5b36-4b8d-accc-067fab8a8efc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:03.933791 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74cd218-5b36-4b8d-accc-067fab8a8efc admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:03.952860 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1013/2039] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:03 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:03.954302 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb36a1f9-4061-47a2-a75f-359987400d17 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:03.965938 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb36a1f9-4061-47a2-a75f-359987400d17 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:03.971614 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145856bc-3dc9-49c2-a426-2df881e9be29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:03.972325 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145856bc-3dc9-49c2-a426-2df881e9be29 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:03.972611 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145856bc-3dc9-49c2-a426-2df881e9be29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:03.972883 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145856bc-3dc9-49c2-a426-2df881e9be29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:03.973526 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145856bc-3dc9-49c2-a426-2df881e9be29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:03.975189 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145856bc-3dc9-49c2-a426-2df881e9be29 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:03.976723 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:03.993241 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74cd218-5b36-4b8d-accc-067fab8a8efc admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:03.994381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1027/2040] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:03 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:04.003806 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74cd218-5b36-4b8d-accc-067fab8a8efc admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:04.009247 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c5cf6f-f64e-449b-81ec-dba15fec5cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:04.009837 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5cf6f-f64e-449b-81ec-dba15fec5cc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:04.010138 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5cf6f-f64e-449b-81ec-dba15fec5cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:04.010404 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5cf6f-f64e-449b-81ec-dba15fec5cc6 admin admin] PATH_INFO: `/v3/projects/4f8cc67c7f404dd0b6707d35c505541d/users/ee623606e2214b5d8ec00c2a60cabc53/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:04.028820 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c5cf6f-f64e-449b-81ec-dba15fec5cc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f8cc67c7f404dd0b6707d35c505541d, user_id=ee623606e2214b5d8ec00c2a60cabc53, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:04.031484 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c5cf6f-f64e-449b-81ec-dba15fec5cc6 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:04.059023 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1014/2041] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:03 2022] PUT /identity/v3/projects/4f8cc67c7f404dd0b6707d35c505541d/users/ee623606e2214b5d8ec00c2a60cabc53/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:43:04.130927 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b1e02-bcfb-4d88-a9e0-609d8dcb508b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:04.131193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b1e02-bcfb-4d88-a9e0-609d8dcb508b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:04.131450 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b1e02-bcfb-4d88-a9e0-609d8dcb508b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:04.189355 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-452b1e02-bcfb-4d88-a9e0-609d8dcb508b None None] MFA Rules not processed for user `ee623606e2214b5d8ec00c2a60cabc53`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:04.190837 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-452b1e02-bcfb-4d88-a9e0-609d8dcb508b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:04.312841 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1028/2042] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:43:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:04.316840 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c5cf6f-f64e-449b-81ec-dba15fec5cc6 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:04.437016 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c5cf6f-f64e-449b-81ec-dba15fec5cc6 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:04.444431 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ddeba-e540-4726-ba92-f79fa7a225f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:04.444431 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ddeba-e540-4726-ba92-f79fa7a225f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:04.444431 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ddeba-e540-4726-ba92-f79fa7a225f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:04.444431 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ddeba-e540-4726-ba92-f79fa7a225f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:04.444829 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510ddeba-e540-4726-ba92-f79fa7a225f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:04.446197 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510ddeba-e540-4726-ba92-f79fa7a225f5 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:04.457258 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:04.487968 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1015/2043] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:43:04 2022] POST /identity/v3/projects => generated 375 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:04.489595 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452b1e02-bcfb-4d88-a9e0-609d8dcb508b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:04.507069 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452b1e02-bcfb-4d88-a9e0-609d8dcb508b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:04.583279 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364a249-1c84-418c-af0a-aed0bc0f8aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:04.584284 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364a249-1c84-418c-af0a-aed0bc0f8aa1 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:04.584284 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364a249-1c84-418c-af0a-aed0bc0f8aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:04.584508 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364a249-1c84-418c-af0a-aed0bc0f8aa1 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:04.585137 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7364a249-1c84-418c-af0a-aed0bc0f8aa1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:04.587117 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7364a249-1c84-418c-af0a-aed0bc0f8aa1 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:04.614221 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1029/2044] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:43:04 2022] POST /identity/v3/users => generated 294 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:43:04.618592 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ddeba-e540-4726-ba92-f79fa7a225f5 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:04.636813 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ddeba-e540-4726-ba92-f79fa7a225f5 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:04.641676 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8029ca-4bd2-48bc-a7a3-2aaae889a86e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:04.642399 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8029ca-4bd2-48bc-a7a3-2aaae889a86e admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:04.642676 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8029ca-4bd2-48bc-a7a3-2aaae889a86e admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:04.642946 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8029ca-4bd2-48bc-a7a3-2aaae889a86e admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:04.643592 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8029ca-4bd2-48bc-a7a3-2aaae889a86e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:04.645378 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8029ca-4bd2-48bc-a7a3-2aaae889a86e admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:04.658822 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1016/2045] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:04 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:43:04.659903 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364a249-1c84-418c-af0a-aed0bc0f8aa1 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:04.678870 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364a249-1c84-418c-af0a-aed0bc0f8aa1 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:04.685991 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7909971d-a1c9-4af7-a56e-211f61b49fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:04.686694 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7909971d-a1c9-4af7-a56e-211f61b49fdf admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:04.686973 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7909971d-a1c9-4af7-a56e-211f61b49fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:04.687376 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7909971d-a1c9-4af7-a56e-211f61b49fdf admin admin] PATH_INFO: `/v3/projects/ec15ff5a45b24d67bc8e933c63f3e3aa/users/9242e846e0bc4b6f8e88bdc320c82f21/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:04.697530 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8029ca-4bd2-48bc-a7a3-2aaae889a86e admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:04.699838 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-da8029ca-4bd2-48bc-a7a3-2aaae889a86e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:04.705507 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7909971d-a1c9-4af7-a56e-211f61b49fdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec15ff5a45b24d67bc8e933c63f3e3aa, user_id=9242e846e0bc4b6f8e88bdc320c82f21, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:04.708173 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7909971d-a1c9-4af7-a56e-211f61b49fdf admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:04.714603 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8029ca-4bd2-48bc-a7a3-2aaae889a86e admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:04.720600 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c931be6e-cd32-4585-979b-17b9a7cae9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:04.722108 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931be6e-cd32-4585-979b-17b9a7cae9d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:04.722310 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931be6e-cd32-4585-979b-17b9a7cae9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:04.722526 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931be6e-cd32-4585-979b-17b9a7cae9d3 admin admin] PATH_INFO: `/v3/users/5cbcfa011bb342aaaaca09b8060d0b6a` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:04.723944 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1030/2046] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:04 2022] PUT /identity/v3/projects/ec15ff5a45b24d67bc8e933c63f3e3aa/users/9242e846e0bc4b6f8e88bdc320c82f21/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:43:04.726877 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c931be6e-cd32-4585-979b-17b9a7cae9d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cbcfa011bb342aaaaca09b8060d0b6a)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:04.726877 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c931be6e-cd32-4585-979b-17b9a7cae9d3 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:04.728733 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7909971d-a1c9-4af7-a56e-211f61b49fdf admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:04.787832 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:04.788829 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c931be6e-cd32-4585-979b-17b9a7cae9d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cbcfa011bb342aaaaca09b8060d0b6a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:04.798217 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c931be6e-cd32-4585-979b-17b9a7cae9d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cbcfa011bb342aaaaca09b8060d0b6a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:04.799179 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c931be6e-cd32-4585-979b-17b9a7cae9d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cbcfa011bb342aaaaca09b8060d0b6a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:04.800485 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7909971d-a1c9-4af7-a56e-211f61b49fdf admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:04.809307 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb63093-367b-4132-a8ac-a92297a4edd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:04.809551 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c931be6e-cd32-4585-979b-17b9a7cae9d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cbcfa011bb342aaaaca09b8060d0b6a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:04.809951 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb63093-367b-4132-a8ac-a92297a4edd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:04.810231 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb63093-367b-4132-a8ac-a92297a4edd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:04.810496 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb63093-367b-4132-a8ac-a92297a4edd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:04.810769 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c931be6e-cd32-4585-979b-17b9a7cae9d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cbcfa011bb342aaaaca09b8060d0b6a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:04.811192 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb63093-367b-4132-a8ac-a92297a4edd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:04.812793 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb63093-367b-4132-a8ac-a92297a4edd0 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:04.830443 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1031/2047] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:04 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:43:04.831527 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb63093-367b-4132-a8ac-a92297a4edd0 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:04.833260 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-beb63093-367b-4132-a8ac-a92297a4edd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:04.833762 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1017/2048] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:43:04 2022] DELETE /identity/v3/users/5cbcfa011bb342aaaaca09b8060d0b6a => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:43:04.838721 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a21489-4f97-4965-894e-160606de3264 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:04.839201 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a21489-4f97-4965-894e-160606de3264 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:04.839611 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a21489-4f97-4965-894e-160606de3264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:04.849907 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb63093-367b-4132-a8ac-a92297a4edd0 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:04.862830 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-25a21489-4f97-4965-894e-160606de3264 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:04.864669 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-25a21489-4f97-4965-894e-160606de3264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:04.924364 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f453c79-07a1-429d-95a4-efa202db87a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:04.925297 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f453c79-07a1-429d-95a4-efa202db87a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:04.925643 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f453c79-07a1-429d-95a4-efa202db87a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:04.926338 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f453c79-07a1-429d-95a4-efa202db87a7 admin admin] PATH_INFO: `/v3/projects/ec15ff5a45b24d67bc8e933c63f3e3aa/users/9242e846e0bc4b6f8e88bdc320c82f21/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:04.929049 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f453c79-07a1-429d-95a4-efa202db87a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec15ff5a45b24d67bc8e933c63f3e3aa, user_id=9242e846e0bc4b6f8e88bdc320c82f21, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:04.932304 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f453c79-07a1-429d-95a4-efa202db87a7 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:04.937773 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1018/2049] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:43:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:04.947505 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a21489-4f97-4965-894e-160606de3264 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:04.949061 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-25a21489-4f97-4965-894e-160606de3264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:04.949757 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1032/2050] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:04 2022] PUT /identity/v3/projects/ec15ff5a45b24d67bc8e933c63f3e3aa/users/9242e846e0bc4b6f8e88bdc320c82f21/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:04.960293 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c6ce9-4b0a-4ee4-9caf-72a72bc0646b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:04.960668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c6ce9-4b0a-4ee4-9caf-72a72bc0646b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:04.961246 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c6ce9-4b0a-4ee4-9caf-72a72bc0646b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:05.008448 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-633c6ce9-4b0a-4ee4-9caf-72a72bc0646b None None] MFA Rules not processed for user `9242e846e0bc4b6f8e88bdc320c82f21`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:05.010347 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-633c6ce9-4b0a-4ee4-9caf-72a72bc0646b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:05.022680 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a21489-4f97-4965-894e-160606de3264 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:05.026523 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:05.077419 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69ffba4-b4f2-4a22-a85f-a856aed679c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:05.078103 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ffba4-b4f2-4a22-a85f-a856aed679c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:05.078468 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ffba4-b4f2-4a22-a85f-a856aed679c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:05.078795 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ffba4-b4f2-4a22-a85f-a856aed679c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:05.095539 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1033/2051] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:43:04 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:43:05.108930 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c6ce9-4b0a-4ee4-9caf-72a72bc0646b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:05.110577 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-633c6ce9-4b0a-4ee4-9caf-72a72bc0646b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:05.158981 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69ffba4-b4f2-4a22-a85f-a856aed679c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:05.160513 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69ffba4-b4f2-4a22-a85f-a856aed679c2 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:05.173714 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1019/2052] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:43:05.182591 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c6ce9-4b0a-4ee4-9caf-72a72bc0646b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:05.186273 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55480d3-df26-4d54-975c-0862a994c18b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:05.186883 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55480d3-df26-4d54-975c-0862a994c18b service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:05.187162 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55480d3-df26-4d54-975c-0862a994c18b service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:05.187431 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55480d3-df26-4d54-975c-0862a994c18b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:05.190505 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a55480d3-df26-4d54-975c-0862a994c18b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:05.211768 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:05.257244 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55480d3-df26-4d54-975c-0862a994c18b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:05.258939 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55480d3-df26-4d54-975c-0862a994c18b service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:05.269834 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1034/2053] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:43:06.061897 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69ffba4-b4f2-4a22-a85f-a856aed679c2 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:06.063747 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e69ffba4-b4f2-4a22-a85f-a856aed679c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:06.077318 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69ffba4-b4f2-4a22-a85f-a856aed679c2 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:06.083382 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214a67ad-56ea-43b8-a683-5679e9ff8afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:06.084061 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a67ad-56ea-43b8-a683-5679e9ff8afa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:06.084329 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a67ad-56ea-43b8-a683-5679e9ff8afa admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:06.084597 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a67ad-56ea-43b8-a683-5679e9ff8afa admin admin] PATH_INFO: `/v3/projects/07612fca609845c4ae38b958c53e0e69` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:06.085890 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214a67ad-56ea-43b8-a683-5679e9ff8afa admin admin] RBAC: Authorizing `identity:delete_project(project_id=07612fca609845c4ae38b958c53e0e69)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:06.088449 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214a67ad-56ea-43b8-a683-5679e9ff8afa admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:06.141460 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-214a67ad-56ea-43b8-a683-5679e9ff8afa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07612fca609845c4ae38b958c53e0e69', 'request_id': 'req-214a67ad-56ea-43b8-a683-5679e9ff8afa'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:06.151789 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-214a67ad-56ea-43b8-a683-5679e9ff8afa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07612fca609845c4ae38b958c53e0e69', 'request_id': 'req-214a67ad-56ea-43b8-a683-5679e9ff8afa'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:06.158618 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-214a67ad-56ea-43b8-a683-5679e9ff8afa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07612fca609845c4ae38b958c53e0e69', 'request_id': 'req-214a67ad-56ea-43b8-a683-5679e9ff8afa'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:06.169071 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-214a67ad-56ea-43b8-a683-5679e9ff8afa admin admin] The token cache is being invalidate because project 07612fca609845c4ae38b958c53e0e69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:43:06.169360 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-214a67ad-56ea-43b8-a683-5679e9ff8afa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:06.170028 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-214a67ad-56ea-43b8-a683-5679e9ff8afa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:06.171512 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1020/2054] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:43:06 2022] DELETE /identity/v3/projects/07612fca609845c4ae38b958c53e0e69 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:06.176550 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a227f-389e-4396-8a1f-3451e51dc268 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:06.177057 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a227f-389e-4396-8a1f-3451e51dc268 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:06.177230 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a227f-389e-4396-8a1f-3451e51dc268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:06.200346 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-b02a227f-389e-4396-8a1f-3451e51dc268 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:06.200346 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b02a227f-389e-4396-8a1f-3451e51dc268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:06.262327 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1035/2055] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:43:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:06.271272 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be2d12d-62b0-4abb-8bd8-c8d9057d3f8d None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:06.271570 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be2d12d-62b0-4abb-8bd8-c8d9057d3f8d None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:06.271834 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be2d12d-62b0-4abb-8bd8-c8d9057d3f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:06.292712 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-9be2d12d-62b0-4abb-8bd8-c8d9057d3f8d None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:06.293994 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-9be2d12d-62b0-4abb-8bd8-c8d9057d3f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:06.301966 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1021/2056] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:43:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:06.306748 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a227f-389e-4396-8a1f-3451e51dc268 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:06.316505 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a227f-389e-4396-8a1f-3451e51dc268 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:06.321564 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16c0869-7da9-4046-8116-64936d254fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:06.322442 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c0869-7da9-4046-8116-64936d254fdc admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:06.322726 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c0869-7da9-4046-8116-64936d254fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:06.323029 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c0869-7da9-4046-8116-64936d254fdc admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:06.323688 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16c0869-7da9-4046-8116-64936d254fdc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:43:06.323988 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16c0869-7da9-4046-8116-64936d254fdc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:06.325602 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16c0869-7da9-4046-8116-64936d254fdc admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:06.340770 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1036/2057] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:43:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:43:06.341912 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be2d12d-62b0-4abb-8bd8-c8d9057d3f8d None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:06.351167 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be2d12d-62b0-4abb-8bd8-c8d9057d3f8d None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:06.357820 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be9edea-0a18-406c-9f0c-64b18a271bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:06.358908 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be9edea-0a18-406c-9f0c-64b18a271bf8 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:06.359367 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be9edea-0a18-406c-9f0c-64b18a271bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:06.359641 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be9edea-0a18-406c-9f0c-64b18a271bf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:06.360287 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be9edea-0a18-406c-9f0c-64b18a271bf8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:06.362240 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be9edea-0a18-406c-9f0c-64b18a271bf8 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:06.398446 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1022/2058] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:43:06 2022] POST /identity/v3/projects => generated 369 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:43:06.402775 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16c0869-7da9-4046-8116-64936d254fdc admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:06.418176 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16c0869-7da9-4046-8116-64936d254fdc admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:06.426945 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210fc53c-f775-40f6-ae48-a82c687a3cdd None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:06.427297 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210fc53c-f775-40f6-ae48-a82c687a3cdd None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:06.427636 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210fc53c-f775-40f6-ae48-a82c687a3cdd None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:06.429330 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1023/2059] 10.222.0.52 () {54 vars in 921 bytes} [Wed Sep 28 15:43:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 28 15:43:06.481008 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fb11e8-3803-4bb5-a435-c39bd8f8afc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:06.481614 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb11e8-3803-4bb5-a435-c39bd8f8afc3 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:06.481934 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb11e8-3803-4bb5-a435-c39bd8f8afc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:06.482209 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb11e8-3803-4bb5-a435-c39bd8f8afc3 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:06.482825 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fb11e8-3803-4bb5-a435-c39bd8f8afc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:06.484860 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fb11e8-3803-4bb5-a435-c39bd8f8afc3 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:06.486093 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d834a3-f74e-48ce-875e-6baf7d5393bf None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:06.486377 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d834a3-f74e-48ce-875e-6baf7d5393bf None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:06.487291 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d834a3-f74e-48ce-875e-6baf7d5393bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:06.511887 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1037/2060] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:43:06 2022] POST /identity/v3/users => generated 285 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:06.515263 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-e2d834a3-f74e-48ce-875e-6baf7d5393bf None None] MFA Rules not processed for user `de934d0105bd4d01ba6c06a4a0d36704`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:06.515700 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fb11e8-3803-4bb5-a435-c39bd8f8afc3 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:06.517215 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e2d834a3-f74e-48ce-875e-6baf7d5393bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:06.525671 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:06.528486 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fb11e8-3803-4bb5-a435-c39bd8f8afc3 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:06.533555 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e691660-7412-4b5c-a917-778e5c0739e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:06.534282 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e691660-7412-4b5c-a917-778e5c0739e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:06.534777 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e691660-7412-4b5c-a917-778e5c0739e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:06.535131 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e691660-7412-4b5c-a917-778e5c0739e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:06.535849 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e691660-7412-4b5c-a917-778e5c0739e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:06.537535 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e691660-7412-4b5c-a917-778e5c0739e9 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:06.550767 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1038/2061] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:06.556319 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e691660-7412-4b5c-a917-778e5c0739e9 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:06.570095 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e691660-7412-4b5c-a917-778e5c0739e9 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:06.581073 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1f9385-7f44-41ae-bce4-60795ea81636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:06.581701 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1f9385-7f44-41ae-bce4-60795ea81636 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:06.582033 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1f9385-7f44-41ae-bce4-60795ea81636 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:06.582304 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1f9385-7f44-41ae-bce4-60795ea81636 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:06.583117 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1f9385-7f44-41ae-bce4-60795ea81636 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:06.584740 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1f9385-7f44-41ae-bce4-60795ea81636 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:06.595762 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1039/2062] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:06 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:06.596797 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1f9385-7f44-41ae-bce4-60795ea81636 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:06.598668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3f1f9385-7f44-41ae-bce4-60795ea81636 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:06.607081 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1024/2063] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Sep 28 15:43:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:06.608023 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d834a3-f74e-48ce-875e-6baf7d5393bf None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:06.612190 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1f9385-7f44-41ae-bce4-60795ea81636 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:06.616908 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e395b1ea-9b99-42d9-8fa9-be2dbba8c688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:06.617696 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395b1ea-9b99-42d9-8fa9-be2dbba8c688 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:06.618513 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395b1ea-9b99-42d9-8fa9-be2dbba8c688 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:06.618746 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d834a3-f74e-48ce-875e-6baf7d5393bf None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:06.618866 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395b1ea-9b99-42d9-8fa9-be2dbba8c688 admin admin] PATH_INFO: `/v3/users/922e0b1b4717422abb6f4e93ea15a86d` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:06.619977 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e395b1ea-9b99-42d9-8fa9-be2dbba8c688 admin admin] RBAC: Authorizing `identity:delete_user(user_id=922e0b1b4717422abb6f4e93ea15a86d)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:06.622109 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e395b1ea-9b99-42d9-8fa9-be2dbba8c688 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:06.624188 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3dd02b-9ea1-43a6-a403-5e13d804342e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:06.624401 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3dd02b-9ea1-43a6-a403-5e13d804342e admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:06.624701 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3dd02b-9ea1-43a6-a403-5e13d804342e admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:06.624974 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3dd02b-9ea1-43a6-a403-5e13d804342e admin admin] PATH_INFO: `/v3/projects/16e81b0c344b46ed88f404017b732e3e/users/ac7f5f01375849558961479868c1b65d/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:06.637338 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:06.643358 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3dd02b-9ea1-43a6-a403-5e13d804342e admin admin] RBAC: Authorizing `identity:create_grant(project_id=16e81b0c344b46ed88f404017b732e3e, user_id=ac7f5f01375849558961479868c1b65d, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:06.646417 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3dd02b-9ea1-43a6-a403-5e13d804342e admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:06.662298 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1025/2064] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:06 2022] PUT /identity/v3/projects/16e81b0c344b46ed88f404017b732e3e/users/ac7f5f01375849558961479868c1b65d/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:06.665590 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f5a35-a55e-457e-953c-4177862c1d74 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:06.665590 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f5a35-a55e-457e-953c-4177862c1d74 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:06.665590 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f5a35-a55e-457e-953c-4177862c1d74 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:06.667367 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1026/2065] 10.222.0.52 () {54 vars in 961 bytes} [Wed Sep 28 15:43:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:43:06.675229 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67721a6-0f0f-442e-9b47-3ba8e5a4984b None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:06.675617 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67721a6-0f0f-442e-9b47-3ba8e5a4984b None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:06.676246 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67721a6-0f0f-442e-9b47-3ba8e5a4984b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:06.693004 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e395b1ea-9b99-42d9-8fa9-be2dbba8c688 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '922e0b1b4717422abb6f4e93ea15a86d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:06.693705 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e395b1ea-9b99-42d9-8fa9-be2dbba8c688 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '922e0b1b4717422abb6f4e93ea15a86d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:06.707403 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e395b1ea-9b99-42d9-8fa9-be2dbba8c688 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '922e0b1b4717422abb6f4e93ea15a86d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:06.714800 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-c67721a6-0f0f-442e-9b47-3ba8e5a4984b None None] MFA Rules not processed for user `ac7f5f01375849558961479868c1b65d`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:06.716513 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c67721a6-0f0f-442e-9b47-3ba8e5a4984b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:06.719175 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e395b1ea-9b99-42d9-8fa9-be2dbba8c688 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '922e0b1b4717422abb6f4e93ea15a86d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:06.731587 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e395b1ea-9b99-42d9-8fa9-be2dbba8c688 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '922e0b1b4717422abb6f4e93ea15a86d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:06.756250 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1040/2066] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:43:06 2022] DELETE /identity/v3/users/922e0b1b4717422abb6f4e93ea15a86d => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:06.757758 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dddc8d-6f46-423a-a396-5f35032beef0 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:06.763931 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dddc8d-6f46-423a-a396-5f35032beef0 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:06.764500 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dddc8d-6f46-423a-a396-5f35032beef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:06.810542 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1027/2067] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:43:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:06.826149 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67721a6-0f0f-442e-9b47-3ba8e5a4984b None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:06.826149 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c67721a6-0f0f-442e-9b47-3ba8e5a4984b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:06.841092 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-c2dddc8d-6f46-423a-a396-5f35032beef0 None None] MFA Rules not processed for user `de934d0105bd4d01ba6c06a4a0d36704`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:06.842734 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c2dddc8d-6f46-423a-a396-5f35032beef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:06.917774 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67721a6-0f0f-442e-9b47-3ba8e5a4984b None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:06.952385 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:06.968743 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1041/2068] 10.222.0.52 () {58 vars in 1058 bytes} [Wed Sep 28 15:43:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:06.981354 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dddc8d-6f46-423a-a396-5f35032beef0 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:06.986223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34324cdb-fcef-4ed1-9a30-bbc589bcad64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:06.986898 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34324cdb-fcef-4ed1-9a30-bbc589bcad64 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:06.987281 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34324cdb-fcef-4ed1-9a30-bbc589bcad64 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:06.987824 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34324cdb-fcef-4ed1-9a30-bbc589bcad64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:06.993467 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dddc8d-6f46-423a-a396-5f35032beef0 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:07.003577 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4615fe6f-eb41-4b93-b05f-fb21f134ef0d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de934d0105bd4d01ba6c06a4a0d36704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:07.004275 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615fe6f-eb41-4b93-b05f-fb21f134ef0d service placement] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:07.004560 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615fe6f-eb41-4b93-b05f-fb21f134ef0d service placement] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:07.004837 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615fe6f-eb41-4b93-b05f-fb21f134ef0d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:07.006796 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4615fe6f-eb41-4b93-b05f-fb21f134ef0d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:07.055064 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4615fe6f-eb41-4b93-b05f-fb21f134ef0d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:07.064551 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4615fe6f-eb41-4b93-b05f-fb21f134ef0d service placement] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:07.107184 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1042/2069] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:43:07.129482 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34324cdb-fcef-4ed1-9a30-bbc589bcad64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:07.131197 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34324cdb-fcef-4ed1-9a30-bbc589bcad64 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:07.137756 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4615fe6f-eb41-4b93-b05f-fb21f134ef0d service placement] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:07.139040 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4615fe6f-eb41-4b93-b05f-fb21f134ef0d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:07.151202 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1028/2070] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:06 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:43:07.202218 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34324cdb-fcef-4ed1-9a30-bbc589bcad64 service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:07.204578 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-34324cdb-fcef-4ed1-9a30-bbc589bcad64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:07.255762 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4615fe6f-eb41-4b93-b05f-fb21f134ef0d service placement] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:07.262329 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0d34f0-1d3d-4487-a730-82563d7c679f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:07.263254 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d34f0-1d3d-4487-a730-82563d7c679f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:07.263697 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d34f0-1d3d-4487-a730-82563d7c679f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:07.264141 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d34f0-1d3d-4487-a730-82563d7c679f admin admin] PATH_INFO: `/v3/projects/6822e0b3eedd404895cd4c0c64082344` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:07.265438 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0d34f0-1d3d-4487-a730-82563d7c679f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6822e0b3eedd404895cd4c0c64082344)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:07.267662 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0d34f0-1d3d-4487-a730-82563d7c679f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:07.286589 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:07.338524 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34324cdb-fcef-4ed1-9a30-bbc589bcad64 service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:07.342848 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3d0d34f0-1d3d-4487-a730-82563d7c679f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6822e0b3eedd404895cd4c0c64082344', 'request_id': 'req-3d0d34f0-1d3d-4487-a730-82563d7c679f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:07.349726 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3d0d34f0-1d3d-4487-a730-82563d7c679f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6822e0b3eedd404895cd4c0c64082344', 'request_id': 'req-3d0d34f0-1d3d-4487-a730-82563d7c679f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:07.361430 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3d0d34f0-1d3d-4487-a730-82563d7c679f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6822e0b3eedd404895cd4c0c64082344', 'request_id': 'req-3d0d34f0-1d3d-4487-a730-82563d7c679f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:07.379804 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3d0d34f0-1d3d-4487-a730-82563d7c679f admin admin] The token cache is being invalidate because project 6822e0b3eedd404895cd4c0c64082344 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:43:07.380139 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3d0d34f0-1d3d-4487-a730-82563d7c679f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:07.381434 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3d0d34f0-1d3d-4487-a730-82563d7c679f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:07.387119 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1043/2071] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:43:07 2022] DELETE /identity/v3/projects/6822e0b3eedd404895cd4c0c64082344 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:43:07.388274 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6f5f9-f266-4fc1-acea-ee97d24ba001 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:07.388726 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6f5f9-f266-4fc1-acea-ee97d24ba001 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:07.389135 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc6f5f9-f266-4fc1-acea-ee97d24ba001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:07.416710 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cd42ef-65f5-412e-9974-2dadf5192a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:07.417623 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd42ef-65f5-412e-9974-2dadf5192a2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:07.418279 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd42ef-65f5-412e-9974-2dadf5192a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:07.419312 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd42ef-65f5-412e-9974-2dadf5192a2d admin admin] PATH_INFO: `/v3/users/ac7f5f01375849558961479868c1b65d` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:07.420646 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cd42ef-65f5-412e-9974-2dadf5192a2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac7f5f01375849558961479868c1b65d)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:07.423092 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cd42ef-65f5-412e-9974-2dadf5192a2d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:07.439896 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-1dc6f5f9-f266-4fc1-acea-ee97d24ba001 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:07.441067 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1dc6f5f9-f266-4fc1-acea-ee97d24ba001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:07.449609 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1044/2072] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:43:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:43:07.457137 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ea10e-fcc4-4f1f-90aa-6c13b5f65c8e None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:07.457427 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ea10e-fcc4-4f1f-90aa-6c13b5f65c8e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:07.457698 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ea10e-fcc4-4f1f-90aa-6c13b5f65c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:07.479193 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:07.479319 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-158ea10e-fcc4-4f1f-90aa-6c13b5f65c8e None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:07.480526 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-158ea10e-fcc4-4f1f-90aa-6c13b5f65c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:07.490378 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1045/2073] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:43:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:07.494631 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-83cd42ef-65f5-412e-9974-2dadf5192a2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac7f5f01375849558961479868c1b65d'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:07.495249 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158ea10e-fcc4-4f1f-90aa-6c13b5f65c8e None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:07.503038 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-83cd42ef-65f5-412e-9974-2dadf5192a2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac7f5f01375849558961479868c1b65d'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:07.503735 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-83cd42ef-65f5-412e-9974-2dadf5192a2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac7f5f01375849558961479868c1b65d'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:07.505940 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158ea10e-fcc4-4f1f-90aa-6c13b5f65c8e None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:07.511790 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b023c0d-3b19-43f1-b8ce-09e866ad8945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:07.512404 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b023c0d-3b19-43f1-b8ce-09e866ad8945 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:07.512510 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-83cd42ef-65f5-412e-9974-2dadf5192a2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac7f5f01375849558961479868c1b65d'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:07.512747 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b023c0d-3b19-43f1-b8ce-09e866ad8945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:07.513031 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b023c0d-3b19-43f1-b8ce-09e866ad8945 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:07.513196 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-83cd42ef-65f5-412e-9974-2dadf5192a2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac7f5f01375849558961479868c1b65d'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:07.513721 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b023c0d-3b19-43f1-b8ce-09e866ad8945 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:43:07.514061 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b023c0d-3b19-43f1-b8ce-09e866ad8945 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:07.515686 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b023c0d-3b19-43f1-b8ce-09e866ad8945 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:07.527895 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1046/2074] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:43:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:43:07.532405 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b023c0d-3b19-43f1-b8ce-09e866ad8945 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:07.534333 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2b023c0d-3b19-43f1-b8ce-09e866ad8945 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:07.535980 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1029/2075] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:43:07 2022] DELETE /identity/v3/users/ac7f5f01375849558961479868c1b65d => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:43:07.540524 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4e946-056b-46d8-92fa-9737d30b83c5 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:07.541962 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4e946-056b-46d8-92fa-9737d30b83c5 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:07.543108 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc4e946-056b-46d8-92fa-9737d30b83c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:07.548874 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b023c0d-3b19-43f1-b8ce-09e866ad8945 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:07.564663 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-4bc4e946-056b-46d8-92fa-9737d30b83c5 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:07.565880 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4bc4e946-056b-46d8-92fa-9737d30b83c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:07.608764 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc1356-6ca4-4dce-bd8c-62645cb918d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:07.609376 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc1356-6ca4-4dce-bd8c-62645cb918d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:07.609817 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc1356-6ca4-4dce-bd8c-62645cb918d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:07.610224 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc1356-6ca4-4dce-bd8c-62645cb918d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:07.610857 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbc1356-6ca4-4dce-bd8c-62645cb918d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:07.612767 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbc1356-6ca4-4dce-bd8c-62645cb918d4 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:07.630220 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1030/2076] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:43:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:43:07.640288 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc4e946-056b-46d8-92fa-9737d30b83c5 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:07.642011 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4bc4e946-056b-46d8-92fa-9737d30b83c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:07.643247 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1047/2077] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:43:07 2022] POST /identity/v3/projects => generated 365 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:43:07.647245 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc1356-6ca4-4dce-bd8c-62645cb918d4 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:07.699151 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc4e946-056b-46d8-92fa-9737d30b83c5 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:07.707770 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc1356-6ca4-4dce-bd8c-62645cb918d4 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:07.712929 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2703d616-72bf-404b-a3a7-8d8c79e4ee3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:07.713526 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2703d616-72bf-404b-a3a7-8d8c79e4ee3d admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:07.713800 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2703d616-72bf-404b-a3a7-8d8c79e4ee3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:07.714096 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2703d616-72bf-404b-a3a7-8d8c79e4ee3d admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:07.714701 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2703d616-72bf-404b-a3a7-8d8c79e4ee3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:07.716736 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2703d616-72bf-404b-a3a7-8d8c79e4ee3d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:07.724670 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:07.743302 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1048/2078] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:43:07 2022] POST /identity/v3/users => generated 283 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:43:07.744679 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2703d616-72bf-404b-a3a7-8d8c79e4ee3d admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:07.751732 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4df51-f744-498d-93df-d3a05e9f36f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:07.752348 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4df51-f744-498d-93df-d3a05e9f36f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:07.752753 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4df51-f744-498d-93df-d3a05e9f36f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:07.752884 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4df51-f744-498d-93df-d3a05e9f36f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:07.755008 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2703d616-72bf-404b-a3a7-8d8c79e4ee3d admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:07.759927 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7148b4b5-f707-4b87-b526-a7b627353957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:07.760516 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148b4b5-f707-4b87-b526-a7b627353957 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:07.760789 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148b4b5-f707-4b87-b526-a7b627353957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:07.761057 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7148b4b5-f707-4b87-b526-a7b627353957 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:07.761686 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7148b4b5-f707-4b87-b526-a7b627353957 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:07.762120 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c4df51-f744-498d-93df-d3a05e9f36f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:07.763318 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7148b4b5-f707-4b87-b526-a7b627353957 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:07.763532 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c4df51-f744-498d-93df-d3a05e9f36f7 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:07.773426 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1031/2079] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:43:07.773572 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1049/2080] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:43:07.778136 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4df51-f744-498d-93df-d3a05e9f36f7 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:07.787151 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4df51-f744-498d-93df-d3a05e9f36f7 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:07.792444 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eb85cd-828f-43cd-b9c5-fc7e153386b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:07.793026 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb85cd-828f-43cd-b9c5-fc7e153386b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:07.793297 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb85cd-828f-43cd-b9c5-fc7e153386b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:07.793563 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb85cd-828f-43cd-b9c5-fc7e153386b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:07.794223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eb85cd-828f-43cd-b9c5-fc7e153386b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:07.795809 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eb85cd-828f-43cd-b9c5-fc7e153386b5 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:07.797912 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:07.807534 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1032/2081] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:43:07.811400 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7148b4b5-f707-4b87-b526-a7b627353957 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:07.821528 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7148b4b5-f707-4b87-b526-a7b627353957 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:07.826391 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c5b345-2230-4e3e-be27-799a305a5736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:07.827037 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c5b345-2230-4e3e-be27-799a305a5736 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:07.827321 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c5b345-2230-4e3e-be27-799a305a5736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:07.827594 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c5b345-2230-4e3e-be27-799a305a5736 admin admin] PATH_INFO: `/v3/projects/861484995daa4fac80bd773f9be0e5f1/users/758e648315224011b06248beeca007b6/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:07.843887 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c5b345-2230-4e3e-be27-799a305a5736 admin admin] RBAC: Authorizing `identity:create_grant(project_id=861484995daa4fac80bd773f9be0e5f1, user_id=758e648315224011b06248beeca007b6, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:07.846595 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c5b345-2230-4e3e-be27-799a305a5736 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:07.860702 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1050/2082] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:07 2022] PUT /identity/v3/projects/861484995daa4fac80bd773f9be0e5f1/users/758e648315224011b06248beeca007b6/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:07.871160 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bcffc4-1267-4f50-83e7-edc9f816ed8a None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:07.871504 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bcffc4-1267-4f50-83e7-edc9f816ed8a None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:07.871777 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bcffc4-1267-4f50-83e7-edc9f816ed8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:07.901506 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-a3bcffc4-1267-4f50-83e7-edc9f816ed8a None None] MFA Rules not processed for user `758e648315224011b06248beeca007b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:07.902827 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a3bcffc4-1267-4f50-83e7-edc9f816ed8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:07.989445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1033/2083] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:43:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:08.007416 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c5b345-2230-4e3e-be27-799a305a5736 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:08.009267 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-88c5b345-2230-4e3e-be27-799a305a5736 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:08.066902 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bcffc4-1267-4f50-83e7-edc9f816ed8a None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:08.069412 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a3bcffc4-1267-4f50-83e7-edc9f816ed8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:08.089404 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c5b345-2230-4e3e-be27-799a305a5736 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:08.094206 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4e68f6-4ed4-4519-8e72-c4aba5522080 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:08.094817 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e68f6-4ed4-4519-8e72-c4aba5522080 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:08.095203 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e68f6-4ed4-4519-8e72-c4aba5522080 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:08.095362 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e68f6-4ed4-4519-8e72-c4aba5522080 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:08.104852 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4e68f6-4ed4-4519-8e72-c4aba5522080 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:08.106521 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4e68f6-4ed4-4519-8e72-c4aba5522080 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:08.118195 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1051/2084] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:43:08.130421 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4e68f6-4ed4-4519-8e72-c4aba5522080 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:08.132277 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ba4e68f6-4ed4-4519-8e72-c4aba5522080 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:08.160455 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bcffc4-1267-4f50-83e7-edc9f816ed8a None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:08.166688 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeca7d6-532d-441e-836e-702fb3d52388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:08.167507 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeca7d6-532d-441e-836e-702fb3d52388 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:08.167789 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeca7d6-532d-441e-836e-702fb3d52388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:08.168146 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeca7d6-532d-441e-836e-702fb3d52388 admin admin] PATH_INFO: `/v3/projects/16e81b0c344b46ed88f404017b732e3e` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:08.169541 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeca7d6-532d-441e-836e-702fb3d52388 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16e81b0c344b46ed88f404017b732e3e)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:08.172911 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeca7d6-532d-441e-836e-702fb3d52388 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:08.214663 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4e68f6-4ed4-4519-8e72-c4aba5522080 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:08.232649 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ebeca7d6-532d-441e-836e-702fb3d52388 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16e81b0c344b46ed88f404017b732e3e', 'request_id': 'req-ebeca7d6-532d-441e-836e-702fb3d52388'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:08.240986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ebeca7d6-532d-441e-836e-702fb3d52388 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16e81b0c344b46ed88f404017b732e3e', 'request_id': 'req-ebeca7d6-532d-441e-836e-702fb3d52388'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:08.249186 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ebeca7d6-532d-441e-836e-702fb3d52388 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16e81b0c344b46ed88f404017b732e3e', 'request_id': 'req-ebeca7d6-532d-441e-836e-702fb3d52388'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:08.264282 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ebeca7d6-532d-441e-836e-702fb3d52388 admin admin] The token cache is being invalidate because project 16e81b0c344b46ed88f404017b732e3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:43:08.264617 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ebeca7d6-532d-441e-836e-702fb3d52388 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:08.265569 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ebeca7d6-532d-441e-836e-702fb3d52388 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:08.267384 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1034/2085] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:43:08 2022] DELETE /identity/v3/projects/16e81b0c344b46ed88f404017b732e3e => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:08.281183 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ba4117-a397-4625-9a6a-174de56d1296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:08.282058 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba4117-a397-4625-9a6a-174de56d1296 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:08.282340 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba4117-a397-4625-9a6a-174de56d1296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:08.282611 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba4117-a397-4625-9a6a-174de56d1296 admin admin] PATH_INFO: `/v3/users/ee623606e2214b5d8ec00c2a60cabc53` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:08.283843 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ba4117-a397-4625-9a6a-174de56d1296 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee623606e2214b5d8ec00c2a60cabc53)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:08.285833 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ba4117-a397-4625-9a6a-174de56d1296 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:08.297123 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:08.354679 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-63ba4117-a397-4625-9a6a-174de56d1296 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee623606e2214b5d8ec00c2a60cabc53'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:08.355490 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-63ba4117-a397-4625-9a6a-174de56d1296 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee623606e2214b5d8ec00c2a60cabc53'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:08.369099 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-63ba4117-a397-4625-9a6a-174de56d1296 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee623606e2214b5d8ec00c2a60cabc53'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:08.377793 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-63ba4117-a397-4625-9a6a-174de56d1296 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee623606e2214b5d8ec00c2a60cabc53'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:08.389029 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-63ba4117-a397-4625-9a6a-174de56d1296 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee623606e2214b5d8ec00c2a60cabc53'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:08.396497 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1052/2086] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:43:08 2022] DELETE /identity/v3/users/ee623606e2214b5d8ec00c2a60cabc53 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:08.402287 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb752018-8973-4e18-a91d-14cf13ec0ccd None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:08.402608 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb752018-8973-4e18-a91d-14cf13ec0ccd None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:08.402993 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb752018-8973-4e18-a91d-14cf13ec0ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:08.467310 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-fb752018-8973-4e18-a91d-14cf13ec0ccd None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:08.467310 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-fb752018-8973-4e18-a91d-14cf13ec0ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:08.493342 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:08.573904 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1035/2087] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:43:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:08.581718 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ba4117-a397-4625-9a6a-174de56d1296 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:08.583167 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-63ba4117-a397-4625-9a6a-174de56d1296 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:08.658687 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ba4117-a397-4625-9a6a-174de56d1296 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:08.661665 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327070d3-3c3a-4723-ba13-a092ac6f2394 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:08.662489 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327070d3-3c3a-4723-ba13-a092ac6f2394 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:08.662861 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327070d3-3c3a-4723-ba13-a092ac6f2394 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:08.663203 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327070d3-3c3a-4723-ba13-a092ac6f2394 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:08.691104 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327070d3-3c3a-4723-ba13-a092ac6f2394 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:08.693193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327070d3-3c3a-4723-ba13-a092ac6f2394 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:08.704208 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1053/2088] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:43:08.991533 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb752018-8973-4e18-a91d-14cf13ec0ccd None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:08.993565 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-fb752018-8973-4e18-a91d-14cf13ec0ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:09.010282 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb752018-8973-4e18-a91d-14cf13ec0ccd None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:09.014147 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47098cad-4953-4aea-8d8b-c3ac7558a5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:09.014915 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47098cad-4953-4aea-8d8b-c3ac7558a5b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:09.015105 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47098cad-4953-4aea-8d8b-c3ac7558a5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:09.015376 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47098cad-4953-4aea-8d8b-c3ac7558a5b1 admin admin] PATH_INFO: `/v3/projects/4f8cc67c7f404dd0b6707d35c505541d` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:09.016529 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47098cad-4953-4aea-8d8b-c3ac7558a5b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f8cc67c7f404dd0b6707d35c505541d)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:09.018394 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47098cad-4953-4aea-8d8b-c3ac7558a5b1 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:09.074314 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-47098cad-4953-4aea-8d8b-c3ac7558a5b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f8cc67c7f404dd0b6707d35c505541d', 'request_id': 'req-47098cad-4953-4aea-8d8b-c3ac7558a5b1'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:09.082206 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-47098cad-4953-4aea-8d8b-c3ac7558a5b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f8cc67c7f404dd0b6707d35c505541d', 'request_id': 'req-47098cad-4953-4aea-8d8b-c3ac7558a5b1'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:09.088616 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-47098cad-4953-4aea-8d8b-c3ac7558a5b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f8cc67c7f404dd0b6707d35c505541d', 'request_id': 'req-47098cad-4953-4aea-8d8b-c3ac7558a5b1'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:09.099119 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-47098cad-4953-4aea-8d8b-c3ac7558a5b1 admin admin] The token cache is being invalidate because project 4f8cc67c7f404dd0b6707d35c505541d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:43:09.099459 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-47098cad-4953-4aea-8d8b-c3ac7558a5b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:09.100265 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-47098cad-4953-4aea-8d8b-c3ac7558a5b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:09.101721 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1036/2089] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:43:08 2022] DELETE /identity/v3/projects/4f8cc67c7f404dd0b6707d35c505541d => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:09.106075 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327070d3-3c3a-4723-ba13-a092ac6f2394 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:09.115406 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327070d3-3c3a-4723-ba13-a092ac6f2394 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:09.173704 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cc1557-7f1b-4b82-a43c-921a77b1ca40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:09.174360 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc1557-7f1b-4b82-a43c-921a77b1ca40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:09.174638 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc1557-7f1b-4b82-a43c-921a77b1ca40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:09.174908 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc1557-7f1b-4b82-a43c-921a77b1ca40 admin admin] PATH_INFO: `/v3/users/9242e846e0bc4b6f8e88bdc320c82f21` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:09.176001 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cc1557-7f1b-4b82-a43c-921a77b1ca40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9242e846e0bc4b6f8e88bdc320c82f21)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:09.177915 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cc1557-7f1b-4b82-a43c-921a77b1ca40 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:09.208591 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:09.228268 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-22cc1557-7f1b-4b82-a43c-921a77b1ca40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9242e846e0bc4b6f8e88bdc320c82f21'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:09.228915 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-22cc1557-7f1b-4b82-a43c-921a77b1ca40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9242e846e0bc4b6f8e88bdc320c82f21'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:09.241103 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-22cc1557-7f1b-4b82-a43c-921a77b1ca40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9242e846e0bc4b6f8e88bdc320c82f21'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:09.248551 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-22cc1557-7f1b-4b82-a43c-921a77b1ca40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9242e846e0bc4b6f8e88bdc320c82f21'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:09.255866 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-22cc1557-7f1b-4b82-a43c-921a77b1ca40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9242e846e0bc4b6f8e88bdc320c82f21'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:09.262493 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1054/2090] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:43:09 2022] DELETE /identity/v3/users/9242e846e0bc4b6f8e88bdc320c82f21 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:09.540691 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47098cad-4953-4aea-8d8b-c3ac7558a5b1 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:09.542165 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-47098cad-4953-4aea-8d8b-c3ac7558a5b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:09.561072 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47098cad-4953-4aea-8d8b-c3ac7558a5b1 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:09.622433 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e0b1a-4d0f-4cae-a57f-b737999a628e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:09.623061 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e0b1a-4d0f-4cae-a57f-b737999a628e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:09.623335 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e0b1a-4d0f-4cae-a57f-b737999a628e admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:09.623604 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e0b1a-4d0f-4cae-a57f-b737999a628e admin admin] PATH_INFO: `/v3/projects/ec15ff5a45b24d67bc8e933c63f3e3aa` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:09.624994 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4e0b1a-4d0f-4cae-a57f-b737999a628e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec15ff5a45b24d67bc8e933c63f3e3aa)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:09.626933 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4e0b1a-4d0f-4cae-a57f-b737999a628e admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:09.671490 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:09.675502 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-aa4e0b1a-4d0f-4cae-a57f-b737999a628e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec15ff5a45b24d67bc8e933c63f3e3aa', 'request_id': 'req-aa4e0b1a-4d0f-4cae-a57f-b737999a628e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:09.683856 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-aa4e0b1a-4d0f-4cae-a57f-b737999a628e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec15ff5a45b24d67bc8e933c63f3e3aa', 'request_id': 'req-aa4e0b1a-4d0f-4cae-a57f-b737999a628e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:09.690275 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-aa4e0b1a-4d0f-4cae-a57f-b737999a628e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec15ff5a45b24d67bc8e933c63f3e3aa', 'request_id': 'req-aa4e0b1a-4d0f-4cae-a57f-b737999a628e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:09.701796 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-aa4e0b1a-4d0f-4cae-a57f-b737999a628e admin admin] The token cache is being invalidate because project ec15ff5a45b24d67bc8e933c63f3e3aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:43:09.702114 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-aa4e0b1a-4d0f-4cae-a57f-b737999a628e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:09.702754 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-aa4e0b1a-4d0f-4cae-a57f-b737999a628e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:09.704208 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1037/2091] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:43:09 2022] DELETE /identity/v3/projects/ec15ff5a45b24d67bc8e933c63f3e3aa => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:09.709210 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9addd79-cfa2-4395-9092-86d534740396 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:09.709631 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9addd79-cfa2-4395-9092-86d534740396 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:09.709898 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9addd79-cfa2-4395-9092-86d534740396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:09.733076 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-e9addd79-cfa2-4395-9092-86d534740396 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:09.734336 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e9addd79-cfa2-4395-9092-86d534740396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:09.799380 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1055/2092] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:43:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:09.808631 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f0ade-4dab-4331-8ea1-6b021ac5574c None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:09.808913 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f0ade-4dab-4331-8ea1-6b021ac5574c None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:09.809247 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f0ade-4dab-4331-8ea1-6b021ac5574c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:09.833105 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-3f2f0ade-4dab-4331-8ea1-6b021ac5574c None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:09.834397 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3f2f0ade-4dab-4331-8ea1-6b021ac5574c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:09.848087 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1038/2093] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:43:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:09.852473 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9addd79-cfa2-4395-9092-86d534740396 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:09.862835 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9addd79-cfa2-4395-9092-86d534740396 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:09.868195 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c6ee7-5a60-475c-b02b-7d1091d32604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:09.869023 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c6ee7-5a60-475c-b02b-7d1091d32604 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:09.869375 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c6ee7-5a60-475c-b02b-7d1091d32604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:09.869714 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c6ee7-5a60-475c-b02b-7d1091d32604 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:09.870489 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0c6ee7-5a60-475c-b02b-7d1091d32604 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:43:09.870865 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0c6ee7-5a60-475c-b02b-7d1091d32604 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:09.885132 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0c6ee7-5a60-475c-b02b-7d1091d32604 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:09.889979 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1056/2094] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:43:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:43:09.894286 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2f0ade-4dab-4331-8ea1-6b021ac5574c None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:09.904937 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2f0ade-4dab-4331-8ea1-6b021ac5574c None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:09.910399 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da17b2b-a284-4fe9-8707-be2dc2a834ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:09.910824 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da17b2b-a284-4fe9-8707-be2dc2a834ec admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:09.911202 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da17b2b-a284-4fe9-8707-be2dc2a834ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:09.911568 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da17b2b-a284-4fe9-8707-be2dc2a834ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:09.912403 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da17b2b-a284-4fe9-8707-be2dc2a834ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:09.914537 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da17b2b-a284-4fe9-8707-be2dc2a834ec admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:09.959601 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1039/2095] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:43:09 2022] POST /identity/v3/projects => generated 377 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:09.964173 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c6ee7-5a60-475c-b02b-7d1091d32604 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:09.974157 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c6ee7-5a60-475c-b02b-7d1091d32604 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:10.032851 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a467b27-a8a7-4c76-9f7b-dcc70156807c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:10.033452 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a467b27-a8a7-4c76-9f7b-dcc70156807c admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:10.033732 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a467b27-a8a7-4c76-9f7b-dcc70156807c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:10.034034 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a467b27-a8a7-4c76-9f7b-dcc70156807c admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:10.034663 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a467b27-a8a7-4c76-9f7b-dcc70156807c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:10.036560 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a467b27-a8a7-4c76-9f7b-dcc70156807c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:10.058205 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1057/2096] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:43:09 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:10.062299 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da17b2b-a284-4fe9-8707-be2dc2a834ec admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:10.073214 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da17b2b-a284-4fe9-8707-be2dc2a834ec admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:10.077959 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe4a991-1c2c-490c-9a98-fb1b51ff160d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:10.078549 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe4a991-1c2c-490c-9a98-fb1b51ff160d admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:10.078823 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe4a991-1c2c-490c-9a98-fb1b51ff160d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:10.079091 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe4a991-1c2c-490c-9a98-fb1b51ff160d admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:10.079724 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe4a991-1c2c-490c-9a98-fb1b51ff160d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:10.081291 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe4a991-1c2c-490c-9a98-fb1b51ff160d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:10.091040 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1040/2097] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:10.094974 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a467b27-a8a7-4c76-9f7b-dcc70156807c admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:10.105111 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a467b27-a8a7-4c76-9f7b-dcc70156807c admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:10.109702 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8900ba5-33e3-473a-a8da-29c2d32ddb70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:10.110333 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8900ba5-33e3-473a-a8da-29c2d32ddb70 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:10.110613 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8900ba5-33e3-473a-a8da-29c2d32ddb70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:10.110883 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8900ba5-33e3-473a-a8da-29c2d32ddb70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:10.111514 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8900ba5-33e3-473a-a8da-29c2d32ddb70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:10.113117 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8900ba5-33e3-473a-a8da-29c2d32ddb70 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:10.124156 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1058/2098] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:10.128102 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe4a991-1c2c-490c-9a98-fb1b51ff160d admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:10.138343 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe4a991-1c2c-490c-9a98-fb1b51ff160d admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:10.142846 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241e6ad5-0d29-4fce-baaf-0d39ed262cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:10.143471 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e6ad5-0d29-4fce-baaf-0d39ed262cec admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:10.143748 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e6ad5-0d29-4fce-baaf-0d39ed262cec admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:10.144015 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e6ad5-0d29-4fce-baaf-0d39ed262cec admin admin] PATH_INFO: `/v3/projects/5444fead438a4fecae6454798f33cc8d/users/0cc45a58c5d14af7a72374f3ae2e64fb/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:10.158083 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241e6ad5-0d29-4fce-baaf-0d39ed262cec admin admin] RBAC: Authorizing `identity:create_grant(project_id=5444fead438a4fecae6454798f33cc8d, user_id=0cc45a58c5d14af7a72374f3ae2e64fb, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:10.160778 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241e6ad5-0d29-4fce-baaf-0d39ed262cec admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:10.173087 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1041/2099] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:10 2022] PUT /identity/v3/projects/5444fead438a4fecae6454798f33cc8d/users/0cc45a58c5d14af7a72374f3ae2e64fb/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:10.183503 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e29f2e-4cff-4ff9-b19c-4d80ca29f707 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:10.183789 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e29f2e-4cff-4ff9-b19c-4d80ca29f707 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:10.184060 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e29f2e-4cff-4ff9-b19c-4d80ca29f707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:10.204422 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:10.212567 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-71e29f2e-4cff-4ff9-b19c-4d80ca29f707 None None] MFA Rules not processed for user `0cc45a58c5d14af7a72374f3ae2e64fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:10.214198 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-71e29f2e-4cff-4ff9-b19c-4d80ca29f707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:10.277894 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1059/2100] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:43:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:10.282510 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241e6ad5-0d29-4fce-baaf-0d39ed262cec admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:10.343333 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241e6ad5-0d29-4fce-baaf-0d39ed262cec admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:10.348123 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a03975-5d5f-46c5-9da9-6f9c2759816c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:10.348723 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a03975-5d5f-46c5-9da9-6f9c2759816c admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:10.349002 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a03975-5d5f-46c5-9da9-6f9c2759816c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:10.349273 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a03975-5d5f-46c5-9da9-6f9c2759816c admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:10.349931 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a03975-5d5f-46c5-9da9-6f9c2759816c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:10.353495 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a03975-5d5f-46c5-9da9-6f9c2759816c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:10.379599 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1042/2101] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:43:10 2022] POST /identity/v3/projects => generated 377 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:10.383494 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e29f2e-4cff-4ff9-b19c-4d80ca29f707 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:10.395791 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e29f2e-4cff-4ff9-b19c-4d80ca29f707 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:10.450041 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faba34b-7973-4eba-b13b-4fe2c2cdee53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:10.450574 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faba34b-7973-4eba-b13b-4fe2c2cdee53 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:10.450850 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faba34b-7973-4eba-b13b-4fe2c2cdee53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:10.451116 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faba34b-7973-4eba-b13b-4fe2c2cdee53 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:10.451736 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faba34b-7973-4eba-b13b-4fe2c2cdee53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:10.453644 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faba34b-7973-4eba-b13b-4fe2c2cdee53 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:10.473762 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1060/2102] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:43:10 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:10.478295 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a03975-5d5f-46c5-9da9-6f9c2759816c admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:10.488990 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a03975-5d5f-46c5-9da9-6f9c2759816c admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:10.494235 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e779057e-82b1-4fb9-8824-d250218fea85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:10.494820 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779057e-82b1-4fb9-8824-d250218fea85 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:10.495091 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779057e-82b1-4fb9-8824-d250218fea85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:10.495358 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779057e-82b1-4fb9-8824-d250218fea85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:10.495989 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e779057e-82b1-4fb9-8824-d250218fea85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:10.497568 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e779057e-82b1-4fb9-8824-d250218fea85 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:10.508744 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1043/2103] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:10.512536 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faba34b-7973-4eba-b13b-4fe2c2cdee53 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:10.522333 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faba34b-7973-4eba-b13b-4fe2c2cdee53 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:10.527349 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab2902-972a-478f-8112-ea605d393dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:10.527938 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab2902-972a-478f-8112-ea605d393dc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:10.528221 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab2902-972a-478f-8112-ea605d393dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:10.528496 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab2902-972a-478f-8112-ea605d393dc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:10.529129 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deab2902-972a-478f-8112-ea605d393dc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:10.530759 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deab2902-972a-478f-8112-ea605d393dc4 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:10.540954 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1061/2104] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:10.544822 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e779057e-82b1-4fb9-8824-d250218fea85 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:10.554710 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e779057e-82b1-4fb9-8824-d250218fea85 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:10.559928 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d28f65-b6e8-4b36-ae09-9f25800a9968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:10.560561 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d28f65-b6e8-4b36-ae09-9f25800a9968 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:10.560838 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d28f65-b6e8-4b36-ae09-9f25800a9968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:10.561111 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d28f65-b6e8-4b36-ae09-9f25800a9968 admin admin] PATH_INFO: `/v3/projects/6dc73f22f1f04476a4387bbb70df98c3/users/b342e6a05d4b478ca9a664268b4b28f8/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:10.575484 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d28f65-b6e8-4b36-ae09-9f25800a9968 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dc73f22f1f04476a4387bbb70df98c3, user_id=b342e6a05d4b478ca9a664268b4b28f8, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:10.578139 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d28f65-b6e8-4b36-ae09-9f25800a9968 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:10.590344 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1044/2105] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:10 2022] PUT /identity/v3/projects/6dc73f22f1f04476a4387bbb70df98c3/users/b342e6a05d4b478ca9a664268b4b28f8/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:10.600454 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24565ba-b6e1-4163-811c-2731c81973e2 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:10.600771 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24565ba-b6e1-4163-811c-2731c81973e2 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:10.601050 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24565ba-b6e1-4163-811c-2731c81973e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:10.630702 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-f24565ba-b6e1-4163-811c-2731c81973e2 None None] MFA Rules not processed for user `b342e6a05d4b478ca9a664268b4b28f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:10.631915 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f24565ba-b6e1-4163-811c-2731c81973e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:10.695034 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1062/2106] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:43:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:10.699453 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d28f65-b6e8-4b36-ae09-9f25800a9968 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:10.718773 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:10.762576 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d28f65-b6e8-4b36-ae09-9f25800a9968 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:10.767111 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a17634-99bc-45ed-a3fa-b7031433afb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:10.767703 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a17634-99bc-45ed-a3fa-b7031433afb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:10.767977 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a17634-99bc-45ed-a3fa-b7031433afb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:10.768244 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a17634-99bc-45ed-a3fa-b7031433afb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:10.768855 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a17634-99bc-45ed-a3fa-b7031433afb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:10.770843 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a17634-99bc-45ed-a3fa-b7031433afb8 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:10.797266 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1045/2107] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:43:10 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:10.801503 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24565ba-b6e1-4163-811c-2731c81973e2 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:10.812042 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24565ba-b6e1-4163-811c-2731c81973e2 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:10.816554 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:10.869246 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8c6a1-7150-4ce6-8b1a-266b7470fea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:10.869882 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8c6a1-7150-4ce6-8b1a-266b7470fea8 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:10.870169 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8c6a1-7150-4ce6-8b1a-266b7470fea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:10.870444 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8c6a1-7150-4ce6-8b1a-266b7470fea8 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:10.871070 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b8c6a1-7150-4ce6-8b1a-266b7470fea8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:10.873022 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b8c6a1-7150-4ce6-8b1a-266b7470fea8 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:10.894513 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1063/2108] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:43:10 2022] POST /identity/v3/users => generated 295 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:10.899056 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a17634-99bc-45ed-a3fa-b7031433afb8 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:10.909428 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a17634-99bc-45ed-a3fa-b7031433afb8 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:10.914132 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9af6b7d-3629-4370-9e1e-43009ba8ddb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:10.915253 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af6b7d-3629-4370-9e1e-43009ba8ddb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:10.915525 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af6b7d-3629-4370-9e1e-43009ba8ddb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:10.915790 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af6b7d-3629-4370-9e1e-43009ba8ddb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:10.916423 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9af6b7d-3629-4370-9e1e-43009ba8ddb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:10.918033 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9af6b7d-3629-4370-9e1e-43009ba8ddb6 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:10.928509 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1046/2109] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:10.932983 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8c6a1-7150-4ce6-8b1a-266b7470fea8 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:10.942933 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8c6a1-7150-4ce6-8b1a-266b7470fea8 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:10.947560 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81b8a3c-0eb8-4fd7-ad19-02303f8436c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:10.948214 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b8a3c-0eb8-4fd7-ad19-02303f8436c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:10.948880 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b8a3c-0eb8-4fd7-ad19-02303f8436c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:10.949155 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b8a3c-0eb8-4fd7-ad19-02303f8436c4 admin admin] PATH_INFO: `/v3/projects/198d135ca99b4a158131d164889fe692/users/0ce702fe5e2449dbaee327ce20005dd8/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:10.966469 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81b8a3c-0eb8-4fd7-ad19-02303f8436c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=198d135ca99b4a158131d164889fe692, user_id=0ce702fe5e2449dbaee327ce20005dd8, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:10.969874 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81b8a3c-0eb8-4fd7-ad19-02303f8436c4 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:10.986716 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1064/2110] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:10 2022] PUT /identity/v3/projects/198d135ca99b4a158131d164889fe692/users/0ce702fe5e2449dbaee327ce20005dd8/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:10.990849 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9af6b7d-3629-4370-9e1e-43009ba8ddb6 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:11.003158 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9af6b7d-3629-4370-9e1e-43009ba8ddb6 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:11.067508 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd5290-381d-479a-b643-b28027ae2673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:11.067657 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd5290-381d-479a-b643-b28027ae2673 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:11.067806 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd5290-381d-479a-b643-b28027ae2673 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:11.068073 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd5290-381d-479a-b643-b28027ae2673 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:11.068705 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cd5290-381d-479a-b643-b28027ae2673 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:11.070313 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cd5290-381d-479a-b643-b28027ae2673 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:11.079964 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1047/2111] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:10 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:11.084237 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81b8a3c-0eb8-4fd7-ad19-02303f8436c4 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:11.094909 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81b8a3c-0eb8-4fd7-ad19-02303f8436c4 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:11.099458 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32144a-34e6-4f04-87ec-670ebe141e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:11.100096 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32144a-34e6-4f04-87ec-670ebe141e37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:11.100374 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32144a-34e6-4f04-87ec-670ebe141e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:11.100644 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32144a-34e6-4f04-87ec-670ebe141e37 admin admin] PATH_INFO: `/v3/projects/198d135ca99b4a158131d164889fe692/users/0ce702fe5e2449dbaee327ce20005dd8/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:11.103193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb32144a-34e6-4f04-87ec-670ebe141e37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=198d135ca99b4a158131d164889fe692, user_id=0ce702fe5e2449dbaee327ce20005dd8, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:11.105833 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb32144a-34e6-4f04-87ec-670ebe141e37 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:11.116086 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1065/2112] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:11 2022] PUT /identity/v3/projects/198d135ca99b4a158131d164889fe692/users/0ce702fe5e2449dbaee327ce20005dd8/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:11.127086 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a60a5ab-0bcc-405e-a952-d344367d9aa9 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:11.127365 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a60a5ab-0bcc-405e-a952-d344367d9aa9 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:11.127634 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a60a5ab-0bcc-405e-a952-d344367d9aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:11.155696 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-2a60a5ab-0bcc-405e-a952-d344367d9aa9 None None] MFA Rules not processed for user `0ce702fe5e2449dbaee327ce20005dd8`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:11.156885 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-2a60a5ab-0bcc-405e-a952-d344367d9aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:11.222532 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1048/2113] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:43:11 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:11.233619 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32144a-34e6-4f04-87ec-670ebe141e37 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:11.235342 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-bb32144a-34e6-4f04-87ec-670ebe141e37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:11.291917 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32144a-34e6-4f04-87ec-670ebe141e37 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:11.295544 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26bc161-f29d-4edf-a194-af15f60f9392 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:11.296143 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26bc161-f29d-4edf-a194-af15f60f9392 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:11.296811 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26bc161-f29d-4edf-a194-af15f60f9392 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:11.296811 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26bc161-f29d-4edf-a194-af15f60f9392 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:11.351857 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26bc161-f29d-4edf-a194-af15f60f9392 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:11.353306 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26bc161-f29d-4edf-a194-af15f60f9392 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:11.362112 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1066/2114] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:43:13.467219 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa541e-13ae-4c32-9617-2ac9c136bf2f None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:13.467657 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa541e-13ae-4c32-9617-2ac9c136bf2f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:13.467743 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faa541e-13ae-4c32-9617-2ac9c136bf2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:13.488920 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-1faa541e-13ae-4c32-9617-2ac9c136bf2f None None] MFA Rules not processed for user `c81e5fb007614cf2992f5e02e85d6307`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:13.490161 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-1faa541e-13ae-4c32-9617-2ac9c136bf2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:13.500220 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:13.549811 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1049/2115] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Sep 28 15:43:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:13.591059 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26bc161-f29d-4edf-a194-af15f60f9392 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:13.592493 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e26bc161-f29d-4edf-a194-af15f60f9392 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:13.611992 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:13.664815 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26bc161-f29d-4edf-a194-af15f60f9392 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:13.670030 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd9d0e0-27d7-4ef6-a8e0-3fdd05b8a1ce service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '328051fe7607464f8f5dc0dcc01a817a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:13.670677 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9d0e0-27d7-4ef6-a8e0-3fdd05b8a1ce service swift] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:13.670968 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9d0e0-27d7-4ef6-a8e0-3fdd05b8a1ce service swift] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:13.671241 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9d0e0-27d7-4ef6-a8e0-3fdd05b8a1ce service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:13.680657 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd9d0e0-27d7-4ef6-a8e0-3fdd05b8a1ce service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:13.682265 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd9d0e0-27d7-4ef6-a8e0-3fdd05b8a1ce service swift] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:13.690637 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1067/2116] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 15:43:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 28 15:43:13.956829 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faa541e-13ae-4c32-9617-2ac9c136bf2f None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:13.958611 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-1faa541e-13ae-4c32-9617-2ac9c136bf2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:13.972124 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faa541e-13ae-4c32-9617-2ac9c136bf2f None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:13.976300 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c08b8-3730-42d1-b36d-b8e7fc55ff0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:13.977187 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c08b8-3730-42d1-b36d-b8e7fc55ff0d service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:13.977734 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c08b8-3730-42d1-b36d-b8e7fc55ff0d service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:13.978071 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c08b8-3730-42d1-b36d-b8e7fc55ff0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:14.041582 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555c08b8-3730-42d1-b36d-b8e7fc55ff0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:14.043061 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555c08b8-3730-42d1-b36d-b8e7fc55ff0d service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:14.055740 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1050/2117] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:13 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:43:14.915960 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd9d0e0-27d7-4ef6-a8e0-3fdd05b8a1ce service swift] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:14.933984 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd9d0e0-27d7-4ef6-a8e0-3fdd05b8a1ce service swift] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:14.938857 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f827a1-f96f-4f67-8890-ed8cd4ff6cf0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:14.939989 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f827a1-f96f-4f67-8890-ed8cd4ff6cf0 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:14.940476 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f827a1-f96f-4f67-8890-ed8cd4ff6cf0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:14.940949 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f827a1-f96f-4f67-8890-ed8cd4ff6cf0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:14.962845 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f827a1-f96f-4f67-8890-ed8cd4ff6cf0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:14.964442 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f827a1-f96f-4f67-8890-ed8cd4ff6cf0 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:14.990086 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1068/2118] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:14 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:43:19.149013 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c08b8-3730-42d1-b36d-b8e7fc55ff0d service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:19.219254 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c08b8-3730-42d1-b36d-b8e7fc55ff0d service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:19.224051 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5555b88-a3cf-40a5-af8a-916036f32f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:19.224713 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5555b88-a3cf-40a5-af8a-916036f32f02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:19.225292 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5555b88-a3cf-40a5-af8a-916036f32f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:19.225829 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5555b88-a3cf-40a5-af8a-916036f32f02 admin admin] PATH_INFO: `/v3/users/0cc45a58c5d14af7a72374f3ae2e64fb` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:19.228388 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5555b88-a3cf-40a5-af8a-916036f32f02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cc45a58c5d14af7a72374f3ae2e64fb)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:19.230394 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5555b88-a3cf-40a5-af8a-916036f32f02 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:19.269435 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f827a1-f96f-4f67-8890-ed8cd4ff6cf0 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:19.272711 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b4f827a1-f96f-4f67-8890-ed8cd4ff6cf0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:19.298649 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f827a1-f96f-4f67-8890-ed8cd4ff6cf0 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:19.305913 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-b5555b88-a3cf-40a5-af8a-916036f32f02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cc45a58c5d14af7a72374f3ae2e64fb'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:19.316332 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-b5555b88-a3cf-40a5-af8a-916036f32f02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cc45a58c5d14af7a72374f3ae2e64fb'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:19.317164 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-b5555b88-a3cf-40a5-af8a-916036f32f02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cc45a58c5d14af7a72374f3ae2e64fb'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:19.318271 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e17d6-494a-4b54-acdf-e15d42106eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:19.319070 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e17d6-494a-4b54-acdf-e15d42106eda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:19.319457 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e17d6-494a-4b54-acdf-e15d42106eda admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:19.320148 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e17d6-494a-4b54-acdf-e15d42106eda admin admin] PATH_INFO: `/v3/users/758e648315224011b06248beeca007b6` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:19.321800 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3e17d6-494a-4b54-acdf-e15d42106eda admin admin] RBAC: Authorizing `identity:delete_user(user_id=758e648315224011b06248beeca007b6)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:19.322285 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:19.324477 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3e17d6-494a-4b54-acdf-e15d42106eda admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:19.340913 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-b5555b88-a3cf-40a5-af8a-916036f32f02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cc45a58c5d14af7a72374f3ae2e64fb'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:19.342914 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-b5555b88-a3cf-40a5-af8a-916036f32f02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cc45a58c5d14af7a72374f3ae2e64fb'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:19.376538 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1051/2119] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:43:19 2022] DELETE /identity/v3/users/0cc45a58c5d14af7a72374f3ae2e64fb => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:19.384120 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e474f3d0-4cca-4cec-bbb3-6545cc3b8886 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:19.384789 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e474f3d0-4cca-4cec-bbb3-6545cc3b8886 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:19.385217 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e474f3d0-4cca-4cec-bbb3-6545cc3b8886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:19.405670 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-cf3e17d6-494a-4b54-acdf-e15d42106eda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '758e648315224011b06248beeca007b6'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:19.406955 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-cf3e17d6-494a-4b54-acdf-e15d42106eda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '758e648315224011b06248beeca007b6'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:19.417472 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-e474f3d0-4cca-4cec-bbb3-6545cc3b8886 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:19.419264 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e474f3d0-4cca-4cec-bbb3-6545cc3b8886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:19.459463 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-cf3e17d6-494a-4b54-acdf-e15d42106eda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '758e648315224011b06248beeca007b6'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:19.482576 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-cf3e17d6-494a-4b54-acdf-e15d42106eda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '758e648315224011b06248beeca007b6'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:19.510551 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-cf3e17d6-494a-4b54-acdf-e15d42106eda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '758e648315224011b06248beeca007b6'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:19.521136 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1069/2120] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:43:19 2022] DELETE /identity/v3/users/758e648315224011b06248beeca007b6 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:43:19.527520 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f4ae2-dc76-4e2f-9ce4-37293a5c0dd8 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:19.530823 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1052/2121] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:43:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:19.534431 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f4ae2-dc76-4e2f-9ce4-37293a5c0dd8 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:19.535615 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f4ae2-dc76-4e2f-9ce4-37293a5c0dd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:19.543211 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e474f3d0-4cca-4cec-bbb3-6545cc3b8886 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:19.544937 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e474f3d0-4cca-4cec-bbb3-6545cc3b8886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:19.572396 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-2e1f4ae2-dc76-4e2f-9ce4-37293a5c0dd8 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:19.573641 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2e1f4ae2-dc76-4e2f-9ce4-37293a5c0dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:19.655715 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e474f3d0-4cca-4cec-bbb3-6545cc3b8886 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:19.690827 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1070/2122] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:43:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:19.701250 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f4ae2-dc76-4e2f-9ce4-37293a5c0dd8 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:19.707900 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:19.766685 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b939830-1e34-4107-8025-cf85b273bf7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:19.768876 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b939830-1e34-4107-8025-cf85b273bf7e service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:19.768876 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b939830-1e34-4107-8025-cf85b273bf7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:19.768876 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b939830-1e34-4107-8025-cf85b273bf7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:19.791940 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f4ae2-dc76-4e2f-9ce4-37293a5c0dd8 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:19.796341 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d161ea8d-8ef7-40a4-ad53-e1dbdf8afe27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:19.800662 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b939830-1e34-4107-8025-cf85b273bf7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:19.800662 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b939830-1e34-4107-8025-cf85b273bf7e service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:19.801726 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d161ea8d-8ef7-40a4-ad53-e1dbdf8afe27 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:19.802178 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d161ea8d-8ef7-40a4-ad53-e1dbdf8afe27 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:19.802583 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d161ea8d-8ef7-40a4-ad53-e1dbdf8afe27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:19.828798 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d161ea8d-8ef7-40a4-ad53-e1dbdf8afe27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:19.830623 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d161ea8d-8ef7-40a4-ad53-e1dbdf8afe27 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:19.831841 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1053/2123] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:43:19.868704 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1071/2124] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:43:20.370006 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b939830-1e34-4107-8025-cf85b273bf7e service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:20.372879 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-2b939830-1e34-4107-8025-cf85b273bf7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:20.387286 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b939830-1e34-4107-8025-cf85b273bf7e service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:20.395687 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d161ea8d-8ef7-40a4-ad53-e1dbdf8afe27 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:20.396271 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5856fc65-d3b2-494d-aab3-920ad939d15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:20.396694 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5856fc65-d3b2-494d-aab3-920ad939d15a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:20.396974 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5856fc65-d3b2-494d-aab3-920ad939d15a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:20.397174 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d161ea8d-8ef7-40a4-ad53-e1dbdf8afe27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:20.397281 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5856fc65-d3b2-494d-aab3-920ad939d15a admin admin] PATH_INFO: `/v3/projects/861484995daa4fac80bd773f9be0e5f1` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:20.398961 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5856fc65-d3b2-494d-aab3-920ad939d15a admin admin] RBAC: Authorizing `identity:delete_project(project_id=861484995daa4fac80bd773f9be0e5f1)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:20.400876 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5856fc65-d3b2-494d-aab3-920ad939d15a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:20.435302 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d161ea8d-8ef7-40a4-ad53-e1dbdf8afe27 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:20.449196 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbbfe82-3312-4315-9790-7a4500238576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:20.449835 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbbfe82-3312-4315-9790-7a4500238576 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:20.450136 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbbfe82-3312-4315-9790-7a4500238576 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:20.450404 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbbfe82-3312-4315-9790-7a4500238576 admin admin] PATH_INFO: `/v3/projects/5444fead438a4fecae6454798f33cc8d` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:20.450786 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:DELETE=5,keystone:SELECT=95 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:20.459796 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbbfe82-3312-4315-9790-7a4500238576 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5444fead438a4fecae6454798f33cc8d)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:20.464645 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbbfe82-3312-4315-9790-7a4500238576 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:20.470110 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5856fc65-d3b2-494d-aab3-920ad939d15a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '861484995daa4fac80bd773f9be0e5f1', 'request_id': 'req-5856fc65-d3b2-494d-aab3-920ad939d15a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:20.483362 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5856fc65-d3b2-494d-aab3-920ad939d15a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '861484995daa4fac80bd773f9be0e5f1', 'request_id': 'req-5856fc65-d3b2-494d-aab3-920ad939d15a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:20.495280 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5856fc65-d3b2-494d-aab3-920ad939d15a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '861484995daa4fac80bd773f9be0e5f1', 'request_id': 'req-5856fc65-d3b2-494d-aab3-920ad939d15a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:20.513345 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5856fc65-d3b2-494d-aab3-920ad939d15a admin admin] The token cache is being invalidate because project 861484995daa4fac80bd773f9be0e5f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:43:20.513743 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5856fc65-d3b2-494d-aab3-920ad939d15a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:20.514581 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5856fc65-d3b2-494d-aab3-920ad939d15a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:20.519029 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1054/2125] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:43:20 2022] DELETE /identity/v3/projects/861484995daa4fac80bd773f9be0e5f1 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:20.525696 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e9cad-375c-4bb3-936c-a1ce1f9e55af None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:20.526203 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e9cad-375c-4bb3-936c-a1ce1f9e55af None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:20.527005 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e9cad-375c-4bb3-936c-a1ce1f9e55af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:20.551683 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2cbbfe82-3312-4315-9790-7a4500238576 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5444fead438a4fecae6454798f33cc8d', 'request_id': 'req-2cbbfe82-3312-4315-9790-7a4500238576'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:20.562599 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-c47e9cad-375c-4bb3-936c-a1ce1f9e55af None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:20.562985 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2cbbfe82-3312-4315-9790-7a4500238576 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5444fead438a4fecae6454798f33cc8d', 'request_id': 'req-2cbbfe82-3312-4315-9790-7a4500238576'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:20.563857 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c47e9cad-375c-4bb3-936c-a1ce1f9e55af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:20.575613 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2cbbfe82-3312-4315-9790-7a4500238576 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5444fead438a4fecae6454798f33cc8d', 'request_id': 'req-2cbbfe82-3312-4315-9790-7a4500238576'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:20.594058 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2cbbfe82-3312-4315-9790-7a4500238576 admin admin] The token cache is being invalidate because project 5444fead438a4fecae6454798f33cc8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:43:20.594344 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2cbbfe82-3312-4315-9790-7a4500238576 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:20.595116 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2cbbfe82-3312-4315-9790-7a4500238576 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:20.596797 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1072/2126] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:43:20 2022] DELETE /identity/v3/projects/5444fead438a4fecae6454798f33cc8d => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:20.600972 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbbfe82-3312-4315-9790-7a4500238576 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:20.602572 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2cbbfe82-3312-4315-9790-7a4500238576 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:20.669945 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1055/2127] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:43:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:20.672304 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55a6a3-9f74-4593-b2a3-4bf2200a9422 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:20.672652 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55a6a3-9f74-4593-b2a3-4bf2200a9422 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:20.672984 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55a6a3-9f74-4593-b2a3-4bf2200a9422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:20.696927 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbbfe82-3312-4315-9790-7a4500238576 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:20.702223 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f86f547-3f6f-437d-a505-73f09abcd47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:20.703004 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f86f547-3f6f-437d-a505-73f09abcd47b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:20.703291 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f86f547-3f6f-437d-a505-73f09abcd47b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:20.703718 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f86f547-3f6f-437d-a505-73f09abcd47b admin admin] PATH_INFO: `/v3/users/b342e6a05d4b478ca9a664268b4b28f8` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:20.704875 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f86f547-3f6f-437d-a505-73f09abcd47b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b342e6a05d4b478ca9a664268b4b28f8)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:20.708773 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f86f547-3f6f-437d-a505-73f09abcd47b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:20.731782 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-7c55a6a3-9f74-4593-b2a3-4bf2200a9422 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:20.734872 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7c55a6a3-9f74-4593-b2a3-4bf2200a9422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:20.755011 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:20.759784 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1056/2128] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:43:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:43:20.761373 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c55a6a3-9f74-4593-b2a3-4bf2200a9422 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:20.774275 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c55a6a3-9f74-4593-b2a3-4bf2200a9422 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:20.785501 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f4695-b984-4085-965d-2231f69b08a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:20.786420 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f4695-b984-4085-965d-2231f69b08a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:20.786789 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f4695-b984-4085-965d-2231f69b08a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:20.787153 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89f4695-b984-4085-965d-2231f69b08a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:20.787918 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89f4695-b984-4085-965d-2231f69b08a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:43:20.788225 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9f86f547-3f6f-437d-a505-73f09abcd47b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b342e6a05d4b478ca9a664268b4b28f8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:20.788389 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89f4695-b984-4085-965d-2231f69b08a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:20.789776 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9f86f547-3f6f-437d-a505-73f09abcd47b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b342e6a05d4b478ca9a664268b4b28f8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:20.790367 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89f4695-b984-4085-965d-2231f69b08a2 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:20.806255 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1057/2129] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:43:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:43:20.814203 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f4695-b984-4085-965d-2231f69b08a2 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:20.815170 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9f86f547-3f6f-437d-a505-73f09abcd47b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b342e6a05d4b478ca9a664268b4b28f8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:20.827867 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89f4695-b984-4085-965d-2231f69b08a2 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:20.828763 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9f86f547-3f6f-437d-a505-73f09abcd47b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b342e6a05d4b478ca9a664268b4b28f8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:20.834978 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ca3297-50cb-4739-917e-916af209354b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:20.835736 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca3297-50cb-4739-917e-916af209354b admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:20.835894 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca3297-50cb-4739-917e-916af209354b admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:20.836141 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca3297-50cb-4739-917e-916af209354b admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:20.836827 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ca3297-50cb-4739-917e-916af209354b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:20.838826 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ca3297-50cb-4739-917e-916af209354b admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:20.839278 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9f86f547-3f6f-437d-a505-73f09abcd47b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b342e6a05d4b478ca9a664268b4b28f8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:20.847582 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1073/2130] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:43:20 2022] DELETE /identity/v3/users/b342e6a05d4b478ca9a664268b4b28f8 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:20.881210 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1058/2131] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:43:20 2022] POST /identity/v3/projects => generated 375 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:20.882887 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f86f547-3f6f-437d-a505-73f09abcd47b admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:20.884548 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9f86f547-3f6f-437d-a505-73f09abcd47b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:21.001880 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f86f547-3f6f-437d-a505-73f09abcd47b admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:21.095016 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f7a1c-3677-4a01-9c00-4c73a2b4369f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:21.095953 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f7a1c-3677-4a01-9c00-4c73a2b4369f admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:21.096486 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f7a1c-3677-4a01-9c00-4c73a2b4369f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:21.096915 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f7a1c-3677-4a01-9c00-4c73a2b4369f admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:21.098060 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1f7a1c-3677-4a01-9c00-4c73a2b4369f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:21.100451 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1f7a1c-3677-4a01-9c00-4c73a2b4369f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:21.131738 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1074/2132] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:43:20 2022] POST /identity/v3/users => generated 288 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:21.137191 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ca3297-50cb-4739-917e-916af209354b admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:21.157955 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ca3297-50cb-4739-917e-916af209354b admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:21.171083 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6847502b-abc7-4780-9744-af6f00880930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:21.171710 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6847502b-abc7-4780-9744-af6f00880930 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:21.171982 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6847502b-abc7-4780-9744-af6f00880930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:21.172247 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6847502b-abc7-4780-9744-af6f00880930 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:21.173102 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6847502b-abc7-4780-9744-af6f00880930 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:21.174869 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6847502b-abc7-4780-9744-af6f00880930 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:21.188412 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1059/2133] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:21 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:21.193457 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f7a1c-3677-4a01-9c00-4c73a2b4369f admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:21.214584 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f7a1c-3677-4a01-9c00-4c73a2b4369f admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:21.219056 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf95625-f63d-4fb1-8b6c-f4e295022931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:21.219668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf95625-f63d-4fb1-8b6c-f4e295022931 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:21.219947 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf95625-f63d-4fb1-8b6c-f4e295022931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:21.220221 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf95625-f63d-4fb1-8b6c-f4e295022931 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:21.220865 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf95625-f63d-4fb1-8b6c-f4e295022931 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:21.222814 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf95625-f63d-4fb1-8b6c-f4e295022931 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:21.239232 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6847502b-abc7-4780-9744-af6f00880930 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:21.249224 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1075/2134] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:21 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:21.254249 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6847502b-abc7-4780-9744-af6f00880930 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:21.259893 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1efa1b-53e9-402a-9f2b-247144ff3738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:21.260641 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1efa1b-53e9-402a-9f2b-247144ff3738 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:21.260984 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1efa1b-53e9-402a-9f2b-247144ff3738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:21.261321 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1efa1b-53e9-402a-9f2b-247144ff3738 admin admin] PATH_INFO: `/v3/projects/4983c88b65074f7c9502ace7f4068199/users/e422d6fc530c40c2b206194f190b0738/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:21.283926 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf95625-f63d-4fb1-8b6c-f4e295022931 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:21.285700 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-aaf95625-f63d-4fb1-8b6c-f4e295022931 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:21.312047 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf95625-f63d-4fb1-8b6c-f4e295022931 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:21.318588 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1efa1b-53e9-402a-9f2b-247144ff3738 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4983c88b65074f7c9502ace7f4068199, user_id=e422d6fc530c40c2b206194f190b0738, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:21.321817 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1efa1b-53e9-402a-9f2b-247144ff3738 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:21.327618 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a396674-59f3-4e43-b31d-21a3d23f4c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:21.328356 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a396674-59f3-4e43-b31d-21a3d23f4c40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:21.328531 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a396674-59f3-4e43-b31d-21a3d23f4c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:21.328811 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a396674-59f3-4e43-b31d-21a3d23f4c40 admin admin] PATH_INFO: `/v3/projects/6dc73f22f1f04476a4387bbb70df98c3` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:21.330508 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a396674-59f3-4e43-b31d-21a3d23f4c40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6dc73f22f1f04476a4387bbb70df98c3)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:21.332701 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a396674-59f3-4e43-b31d-21a3d23f4c40 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:21.347447 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1060/2135] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:21 2022] PUT /identity/v3/projects/4983c88b65074f7c9502ace7f4068199/users/e422d6fc530c40c2b206194f190b0738/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:43:21.360308 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe0c0d-d204-4c6b-8c4b-e1c796d2c59c None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:21.360856 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe0c0d-d204-4c6b-8c4b-e1c796d2c59c None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:21.361608 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe0c0d-d204-4c6b-8c4b-e1c796d2c59c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:21.396647 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:21.408263 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-18fe0c0d-d204-4c6b-8c4b-e1c796d2c59c None None] MFA Rules not processed for user `e422d6fc530c40c2b206194f190b0738`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:21.410892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-18fe0c0d-d204-4c6b-8c4b-e1c796d2c59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:21.420655 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9a396674-59f3-4e43-b31d-21a3d23f4c40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6dc73f22f1f04476a4387bbb70df98c3', 'request_id': 'req-9a396674-59f3-4e43-b31d-21a3d23f4c40'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:21.440472 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9a396674-59f3-4e43-b31d-21a3d23f4c40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6dc73f22f1f04476a4387bbb70df98c3', 'request_id': 'req-9a396674-59f3-4e43-b31d-21a3d23f4c40'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:21.453718 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9a396674-59f3-4e43-b31d-21a3d23f4c40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6dc73f22f1f04476a4387bbb70df98c3', 'request_id': 'req-9a396674-59f3-4e43-b31d-21a3d23f4c40'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:21.457443 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:21.478365 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9a396674-59f3-4e43-b31d-21a3d23f4c40 admin admin] The token cache is being invalidate because project 6dc73f22f1f04476a4387bbb70df98c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:43:21.478960 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9a396674-59f3-4e43-b31d-21a3d23f4c40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:21.479744 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9a396674-59f3-4e43-b31d-21a3d23f4c40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:21.486524 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1076/2136] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:43:21 2022] DELETE /identity/v3/projects/6dc73f22f1f04476a4387bbb70df98c3 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:21.486524 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a396674-59f3-4e43-b31d-21a3d23f4c40 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:21.487878 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9a396674-59f3-4e43-b31d-21a3d23f4c40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:21.519700 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1061/2137] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:43:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:21.524952 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe0c0d-d204-4c6b-8c4b-e1c796d2c59c None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:21.526418 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-18fe0c0d-d204-4c6b-8c4b-e1c796d2c59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:21.569098 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a396674-59f3-4e43-b31d-21a3d23f4c40 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:21.575256 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a215ff-c7d6-44fc-8c4c-f1cd2a0368b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:21.576130 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a215ff-c7d6-44fc-8c4c-f1cd2a0368b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:21.576514 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a215ff-c7d6-44fc-8c4c-f1cd2a0368b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:21.576905 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a215ff-c7d6-44fc-8c4c-f1cd2a0368b8 admin admin] PATH_INFO: `/v3/users/0ce702fe5e2449dbaee327ce20005dd8` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:21.578248 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a215ff-c7d6-44fc-8c4c-f1cd2a0368b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ce702fe5e2449dbaee327ce20005dd8)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:21.581041 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a215ff-c7d6-44fc-8c4c-f1cd2a0368b8 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:21.658526 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b9a215ff-c7d6-44fc-8c4c-f1cd2a0368b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ce702fe5e2449dbaee327ce20005dd8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:21.659240 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b9a215ff-c7d6-44fc-8c4c-f1cd2a0368b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ce702fe5e2449dbaee327ce20005dd8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:21.660599 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe0c0d-d204-4c6b-8c4b-e1c796d2c59c None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:21.670435 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8999d12e-04cf-4acf-b34f-25c89289ab52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:21.672486 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8999d12e-04cf-4acf-b34f-25c89289ab52 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:21.673568 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b9a215ff-c7d6-44fc-8c4c-f1cd2a0368b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ce702fe5e2449dbaee327ce20005dd8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:21.674040 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8999d12e-04cf-4acf-b34f-25c89289ab52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:21.674469 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8999d12e-04cf-4acf-b34f-25c89289ab52 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:21.676637 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8999d12e-04cf-4acf-b34f-25c89289ab52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:21.679960 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8999d12e-04cf-4acf-b34f-25c89289ab52 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:21.684998 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b9a215ff-c7d6-44fc-8c4c-f1cd2a0368b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ce702fe5e2449dbaee327ce20005dd8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:21.697568 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b9a215ff-c7d6-44fc-8c4c-f1cd2a0368b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ce702fe5e2449dbaee327ce20005dd8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:21.708284 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1077/2138] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:43:21 2022] DELETE /identity/v3/users/0ce702fe5e2449dbaee327ce20005dd8 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:21.729261 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1062/2139] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:43:21 2022] POST /identity/v3/projects => generated 377 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:21.733822 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a215ff-c7d6-44fc-8c4c-f1cd2a0368b8 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:21.735421 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b9a215ff-c7d6-44fc-8c4c-f1cd2a0368b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:21.814659 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:21.849630 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a215ff-c7d6-44fc-8c4c-f1cd2a0368b8 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:21.927892 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46bbe07-a48a-4672-a618-c62ea213f82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:21.928755 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46bbe07-a48a-4672-a618-c62ea213f82e admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:21.929864 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46bbe07-a48a-4672-a618-c62ea213f82e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:21.930438 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46bbe07-a48a-4672-a618-c62ea213f82e admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:21.931643 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46bbe07-a48a-4672-a618-c62ea213f82e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:21.934669 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46bbe07-a48a-4672-a618-c62ea213f82e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:21.979703 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1078/2140] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:43:21 2022] POST /identity/v3/users => generated 295 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:21.980709 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8999d12e-04cf-4acf-b34f-25c89289ab52 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:21.996651 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8999d12e-04cf-4acf-b34f-25c89289ab52 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:22.008628 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92772c81-a363-4fcd-bd6f-a33a80a51d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:22.010398 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92772c81-a363-4fcd-bd6f-a33a80a51d4c admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:22.015057 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92772c81-a363-4fcd-bd6f-a33a80a51d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:22.017478 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92772c81-a363-4fcd-bd6f-a33a80a51d4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:22.018749 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92772c81-a363-4fcd-bd6f-a33a80a51d4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:22.020911 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92772c81-a363-4fcd-bd6f-a33a80a51d4c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:22.038372 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1063/2141] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:21 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:22.039873 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46bbe07-a48a-4672-a618-c62ea213f82e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:22.066322 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46bbe07-a48a-4672-a618-c62ea213f82e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:22.076478 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92772c81-a363-4fcd-bd6f-a33a80a51d4c admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:22.077102 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d7f4a-6a2b-43b2-8820-0ed851d89d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:22.077950 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d7f4a-6a2b-43b2-8820-0ed851d89d40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:22.078379 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-92772c81-a363-4fcd-bd6f-a33a80a51d4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:22.078646 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d7f4a-6a2b-43b2-8820-0ed851d89d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:22.078795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d7f4a-6a2b-43b2-8820-0ed851d89d40 admin admin] PATH_INFO: `/v3/projects/db92a2d11b7a41bbaabd895d04aca595/users/a3f8c027aca74b988c14291527326ef5/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:22.101900 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92772c81-a363-4fcd-bd6f-a33a80a51d4c admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:22.108106 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455d63ae-e8ef-42f2-97be-70f67396cbde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:22.109027 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455d63ae-e8ef-42f2-97be-70f67396cbde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:22.109520 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455d63ae-e8ef-42f2-97be-70f67396cbde admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:22.110014 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5d7f4a-6a2b-43b2-8820-0ed851d89d40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db92a2d11b7a41bbaabd895d04aca595, user_id=a3f8c027aca74b988c14291527326ef5, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:22.110201 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455d63ae-e8ef-42f2-97be-70f67396cbde admin admin] PATH_INFO: `/v3/projects/198d135ca99b4a158131d164889fe692` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:22.111707 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455d63ae-e8ef-42f2-97be-70f67396cbde admin admin] RBAC: Authorizing `identity:delete_project(project_id=198d135ca99b4a158131d164889fe692)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:22.113792 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455d63ae-e8ef-42f2-97be-70f67396cbde admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:22.114844 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5d7f4a-6a2b-43b2-8820-0ed851d89d40 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:22.143602 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1079/2142] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:22 2022] PUT /identity/v3/projects/db92a2d11b7a41bbaabd895d04aca595/users/a3f8c027aca74b988c14291527326ef5/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:43:22.147656 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d7f4a-6a2b-43b2-8820-0ed851d89d40 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:22.177707 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-455d63ae-e8ef-42f2-97be-70f67396cbde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '198d135ca99b4a158131d164889fe692', 'request_id': 'req-455d63ae-e8ef-42f2-97be-70f67396cbde'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:22.190188 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-455d63ae-e8ef-42f2-97be-70f67396cbde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '198d135ca99b4a158131d164889fe692', 'request_id': 'req-455d63ae-e8ef-42f2-97be-70f67396cbde'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:22.198260 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-455d63ae-e8ef-42f2-97be-70f67396cbde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '198d135ca99b4a158131d164889fe692', 'request_id': 'req-455d63ae-e8ef-42f2-97be-70f67396cbde'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:22.214025 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-455d63ae-e8ef-42f2-97be-70f67396cbde admin admin] The token cache is being invalidate because project 198d135ca99b4a158131d164889fe692 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:43:22.214482 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-455d63ae-e8ef-42f2-97be-70f67396cbde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:22.215277 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-455d63ae-e8ef-42f2-97be-70f67396cbde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:22.231142 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1064/2143] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:43:22 2022] DELETE /identity/v3/projects/198d135ca99b4a158131d164889fe692 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:43:22.262237 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5d7f4a-6a2b-43b2-8820-0ed851d89d40 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:22.337356 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:22.349263 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b711a2fb-e9e0-4c08-93a6-ad290ecc7d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:22.350761 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b711a2fb-e9e0-4c08-93a6-ad290ecc7d69 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:22.352046 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b711a2fb-e9e0-4c08-93a6-ad290ecc7d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:22.352540 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b711a2fb-e9e0-4c08-93a6-ad290ecc7d69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:22.353478 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b711a2fb-e9e0-4c08-93a6-ad290ecc7d69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:22.355571 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b711a2fb-e9e0-4c08-93a6-ad290ecc7d69 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:22.373256 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1080/2144] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:22 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:22.377548 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455d63ae-e8ef-42f2-97be-70f67396cbde admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:22.379170 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-455d63ae-e8ef-42f2-97be-70f67396cbde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:22.399160 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455d63ae-e8ef-42f2-97be-70f67396cbde admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:22.405626 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79091dee-a182-4a19-941c-1b59f46e523b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:22.406478 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79091dee-a182-4a19-941c-1b59f46e523b admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:22.406828 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79091dee-a182-4a19-941c-1b59f46e523b admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:22.407304 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79091dee-a182-4a19-941c-1b59f46e523b admin admin] PATH_INFO: `/v3/projects/db92a2d11b7a41bbaabd895d04aca595/users/a3f8c027aca74b988c14291527326ef5/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:22.412480 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79091dee-a182-4a19-941c-1b59f46e523b admin admin] RBAC: Authorizing `identity:create_grant(project_id=db92a2d11b7a41bbaabd895d04aca595, user_id=a3f8c027aca74b988c14291527326ef5, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:22.416080 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79091dee-a182-4a19-941c-1b59f46e523b admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:22.442268 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1065/2145] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:22 2022] PUT /identity/v3/projects/db92a2d11b7a41bbaabd895d04aca595/users/a3f8c027aca74b988c14291527326ef5/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:22.448653 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cccddac-5748-4e55-a48c-02499bb2964a None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:22.449219 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cccddac-5748-4e55-a48c-02499bb2964a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:22.449219 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cccddac-5748-4e55-a48c-02499bb2964a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:22.491081 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-3cccddac-5748-4e55-a48c-02499bb2964a None None] MFA Rules not processed for user `a3f8c027aca74b988c14291527326ef5`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:22.493015 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3cccddac-5748-4e55-a48c-02499bb2964a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:22.580984 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1081/2146] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:43:22 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:22.593707 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79091dee-a182-4a19-941c-1b59f46e523b admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:22.595523 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-79091dee-a182-4a19-941c-1b59f46e523b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:22.616068 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:22.665562 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79091dee-a182-4a19-941c-1b59f46e523b admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:22.669647 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df87e9-21f7-4af0-83ed-f12892b01606 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:22.670367 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df87e9-21f7-4af0-83ed-f12892b01606 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:22.670720 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df87e9-21f7-4af0-83ed-f12892b01606 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:22.671059 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df87e9-21f7-4af0-83ed-f12892b01606 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:22.673271 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-59df87e9-21f7-4af0-83ed-f12892b01606 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:22.796311 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59df87e9-21f7-4af0-83ed-f12892b01606 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:22.798047 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59df87e9-21f7-4af0-83ed-f12892b01606 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:22.812058 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1066/2147] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:43:28.716228 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cccddac-5748-4e55-a48c-02499bb2964a None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:28.726874 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cccddac-5748-4e55-a48c-02499bb2964a None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:28.732506 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972b20e4-ab26-404f-862c-b410cd277da2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:28.733120 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b20e4-ab26-404f-862c-b410cd277da2 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:28.733402 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b20e4-ab26-404f-862c-b410cd277da2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:28.733677 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b20e4-ab26-404f-862c-b410cd277da2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:28.744186 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972b20e4-ab26-404f-862c-b410cd277da2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:28.745637 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972b20e4-ab26-404f-862c-b410cd277da2 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:28.769933 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1082/2148] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:28 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:43:32.804264 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=41 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:34.128898 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03975c-9817-449d-8c46-ad603e3319fb None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:34.129450 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03975c-9817-449d-8c46-ad603e3319fb None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:34.129450 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03975c-9817-449d-8c46-ad603e3319fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:34.149511 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-df03975c-9817-449d-8c46-ad603e3319fb None None] MFA Rules not processed for user `c81e5fb007614cf2992f5e02e85d6307`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:34.150733 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-df03975c-9817-449d-8c46-ad603e3319fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:34.217565 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1067/2149] 10.222.0.52 () {58 vars in 967 bytes} [Wed Sep 28 15:43:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:43:34.233229 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972b20e4-ab26-404f-862c-b410cd277da2 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:34.234975 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-972b20e4-ab26-404f-862c-b410cd277da2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:34.315945 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972b20e4-ab26-404f-862c-b410cd277da2 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:34.320989 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74091c0-e97b-448a-bd49-dfd2e99cad5e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '328051fe7607464f8f5dc0dcc01a817a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:34.321583 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74091c0-e97b-448a-bd49-dfd2e99cad5e service swift] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:34.321873 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74091c0-e97b-448a-bd49-dfd2e99cad5e service swift] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:34.322160 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74091c0-e97b-448a-bd49-dfd2e99cad5e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:34.330066 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74091c0-e97b-448a-bd49-dfd2e99cad5e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:34.331653 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74091c0-e97b-448a-bd49-dfd2e99cad5e service swift] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:34.340489 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1083/2150] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 15:43:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 15:43:36.374817 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df03975c-9817-449d-8c46-ad603e3319fb None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:36.376640 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-df03975c-9817-449d-8c46-ad603e3319fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:36.443749 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df03975c-9817-449d-8c46-ad603e3319fb None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:36.449062 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5323d261-e57f-439f-8037-d4d9bc50f709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:36.449686 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323d261-e57f-439f-8037-d4d9bc50f709 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:36.449992 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323d261-e57f-439f-8037-d4d9bc50f709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:36.450261 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323d261-e57f-439f-8037-d4d9bc50f709 admin admin] PATH_INFO: `/v3/users/5558f9c2bb37403b9ff03f2be1aa3113` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:36.451399 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5323d261-e57f-439f-8037-d4d9bc50f709 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5558f9c2bb37403b9ff03f2be1aa3113)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:36.453293 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5323d261-e57f-439f-8037-d4d9bc50f709 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:36.505025 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5323d261-e57f-439f-8037-d4d9bc50f709 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5558f9c2bb37403b9ff03f2be1aa3113'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:36.513500 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5323d261-e57f-439f-8037-d4d9bc50f709 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5558f9c2bb37403b9ff03f2be1aa3113'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:36.514251 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5323d261-e57f-439f-8037-d4d9bc50f709 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5558f9c2bb37403b9ff03f2be1aa3113'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:36.521549 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5323d261-e57f-439f-8037-d4d9bc50f709 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5558f9c2bb37403b9ff03f2be1aa3113'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:36.522225 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5323d261-e57f-439f-8037-d4d9bc50f709 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5558f9c2bb37403b9ff03f2be1aa3113'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:36.542070 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1068/2151] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:43:36 2022] DELETE /identity/v3/users/5558f9c2bb37403b9ff03f2be1aa3113 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:36.715166 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74091c0-e97b-448a-bd49-dfd2e99cad5e service swift] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:36.726071 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74091c0-e97b-448a-bd49-dfd2e99cad5e service swift] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:36.776460 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:36.783612 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01780672-c8f2-4e42-98c9-fea16b7d8d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:36.784241 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01780672-c8f2-4e42-98c9-fea16b7d8d27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:36.784521 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01780672-c8f2-4e42-98c9-fea16b7d8d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:36.784795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01780672-c8f2-4e42-98c9-fea16b7d8d27 admin admin] PATH_INFO: `/v3/projects/f9b4928b5e2d4f6286e8e56d35c813a9` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:36.785924 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01780672-c8f2-4e42-98c9-fea16b7d8d27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9b4928b5e2d4f6286e8e56d35c813a9)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:36.788339 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01780672-c8f2-4e42-98c9-fea16b7d8d27 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:36.835357 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-01780672-c8f2-4e42-98c9-fea16b7d8d27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9b4928b5e2d4f6286e8e56d35c813a9', 'request_id': 'req-01780672-c8f2-4e42-98c9-fea16b7d8d27'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:36.841542 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-01780672-c8f2-4e42-98c9-fea16b7d8d27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9b4928b5e2d4f6286e8e56d35c813a9', 'request_id': 'req-01780672-c8f2-4e42-98c9-fea16b7d8d27'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:36.849713 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-01780672-c8f2-4e42-98c9-fea16b7d8d27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9b4928b5e2d4f6286e8e56d35c813a9', 'request_id': 'req-01780672-c8f2-4e42-98c9-fea16b7d8d27'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:36.860503 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-01780672-c8f2-4e42-98c9-fea16b7d8d27 admin admin] The token cache is being invalidate because project f9b4928b5e2d4f6286e8e56d35c813a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:43:36.860788 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-01780672-c8f2-4e42-98c9-fea16b7d8d27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:36.861411 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-01780672-c8f2-4e42-98c9-fea16b7d8d27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:36.862908 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1084/2152] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:43:36 2022] DELETE /identity/v3/projects/f9b4928b5e2d4f6286e8e56d35c813a9 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:36.868316 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621de1be-8514-4c14-95c6-96c473fef095 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:36.868604 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621de1be-8514-4c14-95c6-96c473fef095 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:36.868874 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621de1be-8514-4c14-95c6-96c473fef095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:36.888221 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-621de1be-8514-4c14-95c6-96c473fef095 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:36.889411 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-621de1be-8514-4c14-95c6-96c473fef095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:36.907409 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:36.949196 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1069/2153] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:43:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:36.956636 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb1923e-8427-4838-a550-32041fc96547 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:36.956919 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb1923e-8427-4838-a550-32041fc96547 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:36.957195 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb1923e-8427-4838-a550-32041fc96547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:36.977627 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-edb1923e-8427-4838-a550-32041fc96547 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:36.978851 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-edb1923e-8427-4838-a550-32041fc96547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:36.986546 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1085/2154] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:43:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:36.990532 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621de1be-8514-4c14-95c6-96c473fef095 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:37.000484 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621de1be-8514-4c14-95c6-96c473fef095 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:37.005058 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561bbe6b-3c19-40ff-9d9b-e091291e6b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:37.005644 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561bbe6b-3c19-40ff-9d9b-e091291e6b9a admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:37.005961 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561bbe6b-3c19-40ff-9d9b-e091291e6b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:37.006235 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561bbe6b-3c19-40ff-9d9b-e091291e6b9a admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:37.006873 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561bbe6b-3c19-40ff-9d9b-e091291e6b9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:43:37.007201 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561bbe6b-3c19-40ff-9d9b-e091291e6b9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:37.008776 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561bbe6b-3c19-40ff-9d9b-e091291e6b9a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:37.020237 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1070/2155] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:43:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:43:37.024905 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb1923e-8427-4838-a550-32041fc96547 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:37.034034 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb1923e-8427-4838-a550-32041fc96547 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:37.039272 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33636843-c399-40d5-af55-22904d4d1370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:37.039853 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33636843-c399-40d5-af55-22904d4d1370 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:37.040129 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33636843-c399-40d5-af55-22904d4d1370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:37.040397 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33636843-c399-40d5-af55-22904d4d1370 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:37.041006 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33636843-c399-40d5-af55-22904d4d1370 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:37.042926 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33636843-c399-40d5-af55-22904d4d1370 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:37.072278 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1086/2156] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:43:37 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:37.076219 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561bbe6b-3c19-40ff-9d9b-e091291e6b9a admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:37.086508 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561bbe6b-3c19-40ff-9d9b-e091291e6b9a admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:37.142712 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517f293c-0db3-44e9-a1ac-6136f2973809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:37.143308 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f293c-0db3-44e9-a1ac-6136f2973809 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:37.143583 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f293c-0db3-44e9-a1ac-6136f2973809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:37.143848 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f293c-0db3-44e9-a1ac-6136f2973809 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:37.144477 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517f293c-0db3-44e9-a1ac-6136f2973809 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:37.146391 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517f293c-0db3-44e9-a1ac-6136f2973809 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:37.167576 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1071/2157] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:43:37 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:37.171690 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33636843-c399-40d5-af55-22904d4d1370 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:37.181634 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33636843-c399-40d5-af55-22904d4d1370 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:37.186161 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e158b27-7241-4c62-a80c-48174e75f61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:37.186732 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e158b27-7241-4c62-a80c-48174e75f61b admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:37.187008 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e158b27-7241-4c62-a80c-48174e75f61b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:37.187276 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e158b27-7241-4c62-a80c-48174e75f61b admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:37.187900 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e158b27-7241-4c62-a80c-48174e75f61b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:37.189473 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e158b27-7241-4c62-a80c-48174e75f61b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:37.199102 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1087/2158] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:37 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:37.202865 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517f293c-0db3-44e9-a1ac-6136f2973809 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:37.213766 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517f293c-0db3-44e9-a1ac-6136f2973809 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:37.218841 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d066d0-97d6-495d-befa-aea7e00ffd40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:37.219475 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d066d0-97d6-495d-befa-aea7e00ffd40 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:37.219761 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d066d0-97d6-495d-befa-aea7e00ffd40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:37.220032 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d066d0-97d6-495d-befa-aea7e00ffd40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:37.220693 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d066d0-97d6-495d-befa-aea7e00ffd40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:37.222320 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d066d0-97d6-495d-befa-aea7e00ffd40 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:37.233937 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1072/2159] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:37.238259 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e158b27-7241-4c62-a80c-48174e75f61b admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:37.248418 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e158b27-7241-4c62-a80c-48174e75f61b admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:37.252927 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbffb63-f847-4c9e-a23d-423648376590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:37.253560 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbffb63-f847-4c9e-a23d-423648376590 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:37.253837 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbffb63-f847-4c9e-a23d-423648376590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:37.254254 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbffb63-f847-4c9e-a23d-423648376590 admin admin] PATH_INFO: `/v3/projects/948574e1f3664ae7a78e706c6f780e62/users/571e878f30854e2bbf6132d96fd98e8a/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:37.268334 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbffb63-f847-4c9e-a23d-423648376590 admin admin] RBAC: Authorizing `identity:create_grant(project_id=948574e1f3664ae7a78e706c6f780e62, user_id=571e878f30854e2bbf6132d96fd98e8a, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:37.270993 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbffb63-f847-4c9e-a23d-423648376590 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:37.284735 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1088/2160] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:37 2022] PUT /identity/v3/projects/948574e1f3664ae7a78e706c6f780e62/users/571e878f30854e2bbf6132d96fd98e8a/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:37.295757 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb975e-b21a-4d29-94c3-946e8a4b82f7 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:37.296101 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb975e-b21a-4d29-94c3-946e8a4b82f7 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:37.296462 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb975e-b21a-4d29-94c3-946e8a4b82f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:37.324797 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-cadb975e-b21a-4d29-94c3-946e8a4b82f7 None None] MFA Rules not processed for user `571e878f30854e2bbf6132d96fd98e8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:37.325928 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-cadb975e-b21a-4d29-94c3-946e8a4b82f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:37.392729 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1073/2161] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:43:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:37.406227 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbffb63-f847-4c9e-a23d-423648376590 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:37.408136 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4dbffb63-f847-4c9e-a23d-423648376590 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:37.487938 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbffb63-f847-4c9e-a23d-423648376590 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:37.491594 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a663f69-3b32-48dc-b984-085b2aafb2e1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6a46bbd249c4434ad7d9442754f098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:37.492189 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a663f69-3b32-48dc-b984-085b2aafb2e1 service glance] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:37.492465 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a663f69-3b32-48dc-b984-085b2aafb2e1 service glance] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:37.492733 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a663f69-3b32-48dc-b984-085b2aafb2e1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:37.501404 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a663f69-3b32-48dc-b984-085b2aafb2e1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:37.502846 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a663f69-3b32-48dc-b984-085b2aafb2e1 service glance] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:37.511985 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1089/2162] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:37 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:43:44.279878 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be458ef-aaec-443e-9b44-dce38a7d6094 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:44.280367 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be458ef-aaec-443e-9b44-dce38a7d6094 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:44.280499 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be458ef-aaec-443e-9b44-dce38a7d6094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:44.308260 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-1be458ef-aaec-443e-9b44-dce38a7d6094 None None] MFA Rules not processed for user `f22b26ed7a47480e8448175517170172`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:44.309460 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-1be458ef-aaec-443e-9b44-dce38a7d6094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:44.323416 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:44.369747 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1074/2163] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:43:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:47.507984 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:53.964800 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a663f69-3b32-48dc-b984-085b2aafb2e1 service glance] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:53.966254 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9a663f69-3b32-48dc-b984-085b2aafb2e1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:54.031384 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a663f69-3b32-48dc-b984-085b2aafb2e1 service glance] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:54.036091 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62be2e74-e0c9-472e-bc79-873f95bf7a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:54.036716 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be2e74-e0c9-472e-bc79-873f95bf7a37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:54.036997 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be2e74-e0c9-472e-bc79-873f95bf7a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:54.037269 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be2e74-e0c9-472e-bc79-873f95bf7a37 admin admin] PATH_INFO: `/v3/users/e422d6fc530c40c2b206194f190b0738` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:54.038456 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62be2e74-e0c9-472e-bc79-873f95bf7a37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e422d6fc530c40c2b206194f190b0738)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:54.040408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62be2e74-e0c9-472e-bc79-873f95bf7a37 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:54.090633 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-62be2e74-e0c9-472e-bc79-873f95bf7a37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e422d6fc530c40c2b206194f190b0738'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:54.091211 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-62be2e74-e0c9-472e-bc79-873f95bf7a37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e422d6fc530c40c2b206194f190b0738'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:54.107052 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-62be2e74-e0c9-472e-bc79-873f95bf7a37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e422d6fc530c40c2b206194f190b0738'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:54.116218 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-62be2e74-e0c9-472e-bc79-873f95bf7a37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e422d6fc530c40c2b206194f190b0738'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:54.123835 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-62be2e74-e0c9-472e-bc79-873f95bf7a37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e422d6fc530c40c2b206194f190b0738'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:54.130968 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1090/2164] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:43:53 2022] DELETE /identity/v3/users/e422d6fc530c40c2b206194f190b0738 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:54.136371 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a843c1-ffc5-4548-b955-80e569e4b200 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:54.136987 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a843c1-ffc5-4548-b955-80e569e4b200 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:54.137687 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a843c1-ffc5-4548-b955-80e569e4b200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:54.158856 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-26a843c1-ffc5-4548-b955-80e569e4b200 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:54.160144 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-26a843c1-ffc5-4548-b955-80e569e4b200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:54.221194 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1075/2165] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:43:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:43:54.231007 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62be2e74-e0c9-472e-bc79-873f95bf7a37 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:54.232432 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-62be2e74-e0c9-472e-bc79-873f95bf7a37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:54.305378 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62be2e74-e0c9-472e-bc79-873f95bf7a37 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:54.312657 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09015397-1da0-4ed3-980d-ebcc6f3061ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:54.313308 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09015397-1da0-4ed3-980d-ebcc6f3061ff service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:54.313644 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09015397-1da0-4ed3-980d-ebcc6f3061ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:54.313923 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09015397-1da0-4ed3-980d-ebcc6f3061ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:54.324410 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09015397-1da0-4ed3-980d-ebcc6f3061ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:54.326015 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09015397-1da0-4ed3-980d-ebcc6f3061ff service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:54.337619 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1091/2166] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:43:54.674637 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a843c1-ffc5-4548-b955-80e569e4b200 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:54.676348 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-26a843c1-ffc5-4548-b955-80e569e4b200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:54.706810 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a843c1-ffc5-4548-b955-80e569e4b200 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:54.716442 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbaad98-886b-45c0-9f16-584291fc750a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:54.717076 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbaad98-886b-45c0-9f16-584291fc750a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:54.717799 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbaad98-886b-45c0-9f16-584291fc750a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:54.718216 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbaad98-886b-45c0-9f16-584291fc750a admin admin] PATH_INFO: `/v3/projects/4983c88b65074f7c9502ace7f4068199` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:54.719398 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbaad98-886b-45c0-9f16-584291fc750a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4983c88b65074f7c9502ace7f4068199)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:54.721312 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbaad98-886b-45c0-9f16-584291fc750a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:54.772843 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5bbaad98-886b-45c0-9f16-584291fc750a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4983c88b65074f7c9502ace7f4068199', 'request_id': 'req-5bbaad98-886b-45c0-9f16-584291fc750a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:54.780897 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5bbaad98-886b-45c0-9f16-584291fc750a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4983c88b65074f7c9502ace7f4068199', 'request_id': 'req-5bbaad98-886b-45c0-9f16-584291fc750a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:54.787759 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5bbaad98-886b-45c0-9f16-584291fc750a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4983c88b65074f7c9502ace7f4068199', 'request_id': 'req-5bbaad98-886b-45c0-9f16-584291fc750a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:54.799086 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5bbaad98-886b-45c0-9f16-584291fc750a admin admin] The token cache is being invalidate because project 4983c88b65074f7c9502ace7f4068199 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:43:54.799371 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5bbaad98-886b-45c0-9f16-584291fc750a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:54.800008 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5bbaad98-886b-45c0-9f16-584291fc750a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:54.804962 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1076/2167] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:43:54 2022] DELETE /identity/v3/projects/4983c88b65074f7c9502ace7f4068199 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:54.806221 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09015397-1da0-4ed3-980d-ebcc6f3061ff service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:54.815880 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09015397-1da0-4ed3-980d-ebcc6f3061ff service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:54.834000 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:54.877320 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9dc81a-bd07-4961-bf18-efdb7802270c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:54.878234 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9dc81a-bd07-4961-bf18-efdb7802270c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:54.878517 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9dc81a-bd07-4961-bf18-efdb7802270c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:54.878789 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9dc81a-bd07-4961-bf18-efdb7802270c admin admin] PATH_INFO: `/v3/users/a3f8c027aca74b988c14291527326ef5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:54.879956 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9dc81a-bd07-4961-bf18-efdb7802270c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3f8c027aca74b988c14291527326ef5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:54.881912 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9dc81a-bd07-4961-bf18-efdb7802270c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:54.980818 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-dd9dc81a-bd07-4961-bf18-efdb7802270c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3f8c027aca74b988c14291527326ef5'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:54.982217 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-dd9dc81a-bd07-4961-bf18-efdb7802270c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3f8c027aca74b988c14291527326ef5'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:55.002217 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-dd9dc81a-bd07-4961-bf18-efdb7802270c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3f8c027aca74b988c14291527326ef5'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:55.011104 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-dd9dc81a-bd07-4961-bf18-efdb7802270c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3f8c027aca74b988c14291527326ef5'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:55.018884 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-dd9dc81a-bd07-4961-bf18-efdb7802270c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3f8c027aca74b988c14291527326ef5'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:55.025393 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1092/2168] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:43:54 2022] DELETE /identity/v3/users/a3f8c027aca74b988c14291527326ef5 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:55.312459 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbaad98-886b-45c0-9f16-584291fc750a admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:55.313947 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-5bbaad98-886b-45c0-9f16-584291fc750a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:55.327973 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbaad98-886b-45c0-9f16-584291fc750a admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:55.374069 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:55.388972 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729eab61-872c-41c8-8720-6d3fb299c11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:55.389615 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729eab61-872c-41c8-8720-6d3fb299c11b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:55.389920 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729eab61-872c-41c8-8720-6d3fb299c11b admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:55.390195 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729eab61-872c-41c8-8720-6d3fb299c11b admin admin] PATH_INFO: `/v3/projects/db92a2d11b7a41bbaabd895d04aca595` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:55.391379 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729eab61-872c-41c8-8720-6d3fb299c11b admin admin] RBAC: Authorizing `identity:delete_project(project_id=db92a2d11b7a41bbaabd895d04aca595)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:55.393334 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729eab61-872c-41c8-8720-6d3fb299c11b admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:55.445959 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-729eab61-872c-41c8-8720-6d3fb299c11b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db92a2d11b7a41bbaabd895d04aca595', 'request_id': 'req-729eab61-872c-41c8-8720-6d3fb299c11b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:55.454118 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-729eab61-872c-41c8-8720-6d3fb299c11b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db92a2d11b7a41bbaabd895d04aca595', 'request_id': 'req-729eab61-872c-41c8-8720-6d3fb299c11b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:55.461076 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-729eab61-872c-41c8-8720-6d3fb299c11b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db92a2d11b7a41bbaabd895d04aca595', 'request_id': 'req-729eab61-872c-41c8-8720-6d3fb299c11b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:55.473279 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-729eab61-872c-41c8-8720-6d3fb299c11b admin admin] The token cache is being invalidate because project db92a2d11b7a41bbaabd895d04aca595 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:43:55.473759 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-729eab61-872c-41c8-8720-6d3fb299c11b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:55.474527 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-729eab61-872c-41c8-8720-6d3fb299c11b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:55.476328 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1077/2169] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:43:55 2022] DELETE /identity/v3/projects/db92a2d11b7a41bbaabd895d04aca595 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:55.481611 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46525f8c-80d6-463f-846d-9fee9b427941 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:55.482210 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46525f8c-80d6-463f-846d-9fee9b427941 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:55.482584 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46525f8c-80d6-463f-846d-9fee9b427941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:55.502311 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-46525f8c-80d6-463f-846d-9fee9b427941 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:55.503508 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-46525f8c-80d6-463f-846d-9fee9b427941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:55.570660 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1093/2170] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:43:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:55.578848 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84416c86-5b70-480e-b4e4-454952753283 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:55.579133 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84416c86-5b70-480e-b4e4-454952753283 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:55.579406 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84416c86-5b70-480e-b4e4-454952753283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:55.599699 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-84416c86-5b70-480e-b4e4-454952753283 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:55.600902 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-84416c86-5b70-480e-b4e4-454952753283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:55.608907 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1078/2171] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:43:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:55.613272 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46525f8c-80d6-463f-846d-9fee9b427941 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:55.623419 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46525f8c-80d6-463f-846d-9fee9b427941 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:55.628043 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7148d5-1b8b-42ab-b1a0-925d29ffb28f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:55.628657 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7148d5-1b8b-42ab-b1a0-925d29ffb28f admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:55.628938 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7148d5-1b8b-42ab-b1a0-925d29ffb28f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:55.629212 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7148d5-1b8b-42ab-b1a0-925d29ffb28f admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:55.629870 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7148d5-1b8b-42ab-b1a0-925d29ffb28f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:43:55.630183 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7148d5-1b8b-42ab-b1a0-925d29ffb28f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:55.631778 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7148d5-1b8b-42ab-b1a0-925d29ffb28f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:55.644601 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1094/2172] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:43:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:43:55.649358 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84416c86-5b70-480e-b4e4-454952753283 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:55.659558 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84416c86-5b70-480e-b4e4-454952753283 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:55.664290 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5bd7a9-3897-472a-acf1-c2c5ea5879a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:55.665031 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5bd7a9-3897-472a-acf1-c2c5ea5879a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:55.665170 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5bd7a9-3897-472a-acf1-c2c5ea5879a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:55.665433 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5bd7a9-3897-472a-acf1-c2c5ea5879a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:55.666446 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5bd7a9-3897-472a-acf1-c2c5ea5879a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:55.668424 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5bd7a9-3897-472a-acf1-c2c5ea5879a2 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:55.698134 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1079/2173] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:43:55 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:43:55.702998 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7148d5-1b8b-42ab-b1a0-925d29ffb28f admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:55.722869 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7148d5-1b8b-42ab-b1a0-925d29ffb28f admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:55.752681 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:55.789711 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e480adcb-53e9-4e23-b596-13fa28d38294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:55.790419 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e480adcb-53e9-4e23-b596-13fa28d38294 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:55.790702 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e480adcb-53e9-4e23-b596-13fa28d38294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:55.790975 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e480adcb-53e9-4e23-b596-13fa28d38294 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:55.791602 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e480adcb-53e9-4e23-b596-13fa28d38294 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:55.793532 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e480adcb-53e9-4e23-b596-13fa28d38294 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:55.818222 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1095/2174] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:43:55 2022] POST /identity/v3/users => generated 292 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:55.820464 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5bd7a9-3897-472a-acf1-c2c5ea5879a2 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:55.831174 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5bd7a9-3897-472a-acf1-c2c5ea5879a2 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:55.836098 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c25c1f6-cd93-4827-b8ef-74c4b4a75c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:55.836778 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25c1f6-cd93-4827-b8ef-74c4b4a75c09 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:55.837127 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25c1f6-cd93-4827-b8ef-74c4b4a75c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:55.837448 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25c1f6-cd93-4827-b8ef-74c4b4a75c09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:55.838280 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c25c1f6-cd93-4827-b8ef-74c4b4a75c09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:55.839934 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c25c1f6-cd93-4827-b8ef-74c4b4a75c09 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:55.850765 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1080/2175] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:55.854793 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e480adcb-53e9-4e23-b596-13fa28d38294 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:55.866058 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e480adcb-53e9-4e23-b596-13fa28d38294 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:55.871232 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ffd263-f763-43d9-9d77-9c048e852da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:55.871833 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ffd263-f763-43d9-9d77-9c048e852da4 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:55.872113 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ffd263-f763-43d9-9d77-9c048e852da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:55.872385 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ffd263-f763-43d9-9d77-9c048e852da4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:55.873283 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ffd263-f763-43d9-9d77-9c048e852da4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:55.874926 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ffd263-f763-43d9-9d77-9c048e852da4 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:55.886731 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1096/2176] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:55.890947 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c25c1f6-cd93-4827-b8ef-74c4b4a75c09 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:55.901263 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c25c1f6-cd93-4827-b8ef-74c4b4a75c09 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:55.905909 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb9db37-55d1-40a1-87a7-7044d8cd6a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:55.906562 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9db37-55d1-40a1-87a7-7044d8cd6a71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:55.906837 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9db37-55d1-40a1-87a7-7044d8cd6a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:55.907110 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9db37-55d1-40a1-87a7-7044d8cd6a71 admin admin] PATH_INFO: `/v3/projects/97672b3a5d044644961c1844b3c17357/users/81281c059285485aa4e9ef64d5e112a0/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:55.922543 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb9db37-55d1-40a1-87a7-7044d8cd6a71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97672b3a5d044644961c1844b3c17357, user_id=81281c059285485aa4e9ef64d5e112a0, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:55.925450 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb9db37-55d1-40a1-87a7-7044d8cd6a71 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:55.949439 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1081/2177] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:55 2022] PUT /identity/v3/projects/97672b3a5d044644961c1844b3c17357/users/81281c059285485aa4e9ef64d5e112a0/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:55.952218 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c5920-9b75-4588-a009-bac3e33b3117 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:55.953006 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c5920-9b75-4588-a009-bac3e33b3117 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:55.953006 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051c5920-9b75-4588-a009-bac3e33b3117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:55.984920 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-051c5920-9b75-4588-a009-bac3e33b3117 None None] MFA Rules not processed for user `81281c059285485aa4e9ef64d5e112a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:55.986289 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-051c5920-9b75-4588-a009-bac3e33b3117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:56.053772 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1097/2178] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:43:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:56.059015 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb9db37-55d1-40a1-87a7-7044d8cd6a71 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:56.124072 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb9db37-55d1-40a1-87a7-7044d8cd6a71 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:56.128735 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b205c5-2ea4-4396-8a19-86077ee88fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:56.129366 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b205c5-2ea4-4396-8a19-86077ee88fe5 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:56.129650 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b205c5-2ea4-4396-8a19-86077ee88fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:56.130102 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b205c5-2ea4-4396-8a19-86077ee88fe5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:56.130584 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b205c5-2ea4-4396-8a19-86077ee88fe5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:56.132509 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b205c5-2ea4-4396-8a19-86077ee88fe5 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:56.155268 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:56.161895 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1082/2179] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:43:56 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:43:56.166322 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051c5920-9b75-4588-a009-bac3e33b3117 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:56.176994 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051c5920-9b75-4588-a009-bac3e33b3117 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:56.236643 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d07bad-b033-4c9d-aff1-09cb7899ca6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:56.237253 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d07bad-b033-4c9d-aff1-09cb7899ca6d admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:56.237534 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d07bad-b033-4c9d-aff1-09cb7899ca6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:56.237808 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d07bad-b033-4c9d-aff1-09cb7899ca6d admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:56.238467 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d07bad-b033-4c9d-aff1-09cb7899ca6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:56.240373 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d07bad-b033-4c9d-aff1-09cb7899ca6d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:56.264547 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1098/2180] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:43:56 2022] POST /identity/v3/users => generated 298 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:43:56.268640 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b205c5-2ea4-4396-8a19-86077ee88fe5 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:56.279285 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b205c5-2ea4-4396-8a19-86077ee88fe5 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:56.284100 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd01d8cd-09f4-4553-a700-9acd54d025f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:56.284676 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd01d8cd-09f4-4553-a700-9acd54d025f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:56.284953 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd01d8cd-09f4-4553-a700-9acd54d025f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:56.285223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd01d8cd-09f4-4553-a700-9acd54d025f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:56.285889 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd01d8cd-09f4-4553-a700-9acd54d025f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:56.287474 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd01d8cd-09f4-4553-a700-9acd54d025f1 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:56.301800 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1083/2181] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:56.308093 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d07bad-b033-4c9d-aff1-09cb7899ca6d admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:56.318010 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d07bad-b033-4c9d-aff1-09cb7899ca6d admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:56.322650 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd27bbb1-09c9-4e9a-8689-eed9cf4f0130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:56.323301 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27bbb1-09c9-4e9a-8689-eed9cf4f0130 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:56.323577 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27bbb1-09c9-4e9a-8689-eed9cf4f0130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:56.323959 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd27bbb1-09c9-4e9a-8689-eed9cf4f0130 admin admin] PATH_INFO: `/v3/projects/9f15387d660347b1805a900418d4a6fb/users/ba26cb0db94143c4811889b80b5b54f5/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:56.338409 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd27bbb1-09c9-4e9a-8689-eed9cf4f0130 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f15387d660347b1805a900418d4a6fb, user_id=ba26cb0db94143c4811889b80b5b54f5, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:56.341274 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd27bbb1-09c9-4e9a-8689-eed9cf4f0130 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:56.354953 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1099/2182] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:56 2022] PUT /identity/v3/projects/9f15387d660347b1805a900418d4a6fb/users/ba26cb0db94143c4811889b80b5b54f5/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:43:56.358881 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd01d8cd-09f4-4553-a700-9acd54d025f1 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:56.370230 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd01d8cd-09f4-4553-a700-9acd54d025f1 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:56.445164 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0fc6db-f8e7-4d84-93b5-67bc84649d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:56.446272 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0fc6db-f8e7-4d84-93b5-67bc84649d83 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:56.446749 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0fc6db-f8e7-4d84-93b5-67bc84649d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:56.447184 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0fc6db-f8e7-4d84-93b5-67bc84649d83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:56.447872 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0fc6db-f8e7-4d84-93b5-67bc84649d83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:56.449744 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0fc6db-f8e7-4d84-93b5-67bc84649d83 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:56.467480 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1084/2183] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:56 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:43:56.471883 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd27bbb1-09c9-4e9a-8689-eed9cf4f0130 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:56.478544 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:56.486522 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd27bbb1-09c9-4e9a-8689-eed9cf4f0130 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:56.493647 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f05165-75bb-4ec8-823e-8dde7e7c938e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:56.494347 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f05165-75bb-4ec8-823e-8dde7e7c938e admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:56.494642 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f05165-75bb-4ec8-823e-8dde7e7c938e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:56.494909 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f05165-75bb-4ec8-823e-8dde7e7c938e admin admin] PATH_INFO: `/v3/projects/9f15387d660347b1805a900418d4a6fb/users/ba26cb0db94143c4811889b80b5b54f5/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:56.496812 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f05165-75bb-4ec8-823e-8dde7e7c938e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f15387d660347b1805a900418d4a6fb, user_id=ba26cb0db94143c4811889b80b5b54f5, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:56.499795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f05165-75bb-4ec8-823e-8dde7e7c938e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:56.517909 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1100/2184] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:56 2022] PUT /identity/v3/projects/9f15387d660347b1805a900418d4a6fb/users/ba26cb0db94143c4811889b80b5b54f5/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:56.523118 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa468d-dbaa-42fd-a2b7-9ba1b40883f8 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:56.523404 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa468d-dbaa-42fd-a2b7-9ba1b40883f8 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:56.523720 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa468d-dbaa-42fd-a2b7-9ba1b40883f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:56.557045 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-fdfa468d-dbaa-42fd-a2b7-9ba1b40883f8 None None] MFA Rules not processed for user `ba26cb0db94143c4811889b80b5b54f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:56.558312 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-fdfa468d-dbaa-42fd-a2b7-9ba1b40883f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:56.638737 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1085/2185] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:43:56 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:43:56.651999 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f05165-75bb-4ec8-823e-8dde7e7c938e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:56.653876 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-68f05165-75bb-4ec8-823e-8dde7e7c938e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:56.717319 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f05165-75bb-4ec8-823e-8dde7e7c938e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:56.721003 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546a5274-d7d6-45cb-959c-afe21fe5fc22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:56.721604 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a5274-d7d6-45cb-959c-afe21fe5fc22 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:56.721911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a5274-d7d6-45cb-959c-afe21fe5fc22 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:56.722186 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a5274-d7d6-45cb-959c-afe21fe5fc22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:56.732192 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546a5274-d7d6-45cb-959c-afe21fe5fc22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:56.733630 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546a5274-d7d6-45cb-959c-afe21fe5fc22 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:56.743386 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1101/2186] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:56 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:43:56.844124 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa468d-dbaa-42fd-a2b7-9ba1b40883f8 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:56.853797 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa468d-dbaa-42fd-a2b7-9ba1b40883f8 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:56.857387 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e1b9d-1361-4ee6-8064-037f98fa74d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:56.858021 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e1b9d-1361-4ee6-8064-037f98fa74d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:56.858306 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e1b9d-1361-4ee6-8064-037f98fa74d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:56.858575 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e1b9d-1361-4ee6-8064-037f98fa74d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:56.921012 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9e1b9d-1361-4ee6-8064-037f98fa74d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:56.922681 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9e1b9d-1361-4ee6-8064-037f98fa74d4 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:56.933041 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1086/2187] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:43:57.214938 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546a5274-d7d6-45cb-959c-afe21fe5fc22 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:57.287563 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546a5274-d7d6-45cb-959c-afe21fe5fc22 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:57.296453 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2595d9-5ec7-498e-a8b8-a23c10613245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:57.297090 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2595d9-5ec7-498e-a8b8-a23c10613245 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:57.297443 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2595d9-5ec7-498e-a8b8-a23c10613245 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:57.297623 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2595d9-5ec7-498e-a8b8-a23c10613245 admin admin] PATH_INFO: `/v3/users/81281c059285485aa4e9ef64d5e112a0` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:57.298897 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2595d9-5ec7-498e-a8b8-a23c10613245 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81281c059285485aa4e9ef64d5e112a0)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:57.300990 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2595d9-5ec7-498e-a8b8-a23c10613245 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:57.349835 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ec2595d9-5ec7-498e-a8b8-a23c10613245 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81281c059285485aa4e9ef64d5e112a0'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:57.350645 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ec2595d9-5ec7-498e-a8b8-a23c10613245 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81281c059285485aa4e9ef64d5e112a0'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:57.363985 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ec2595d9-5ec7-498e-a8b8-a23c10613245 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81281c059285485aa4e9ef64d5e112a0'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:57.372420 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ec2595d9-5ec7-498e-a8b8-a23c10613245 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81281c059285485aa4e9ef64d5e112a0'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:57.380295 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ec2595d9-5ec7-498e-a8b8-a23c10613245 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81281c059285485aa4e9ef64d5e112a0'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:57.387007 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1102/2188] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:43:57 2022] DELETE /identity/v3/users/81281c059285485aa4e9ef64d5e112a0 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:57.391293 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaff712-a718-4cdb-8d3a-a9fda632a386 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:57.391737 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaff712-a718-4cdb-8d3a-a9fda632a386 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:57.391895 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaff712-a718-4cdb-8d3a-a9fda632a386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:57.411005 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-ddaff712-a718-4cdb-8d3a-a9fda632a386 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:57.412312 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ddaff712-a718-4cdb-8d3a-a9fda632a386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:57.416448 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:57.477906 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1087/2189] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:43:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:57.487728 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2595d9-5ec7-498e-a8b8-a23c10613245 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:57.489123 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ec2595d9-5ec7-498e-a8b8-a23c10613245 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:57.505301 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:57.556297 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2595d9-5ec7-498e-a8b8-a23c10613245 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:57.560065 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc3124-1193-4125-87e8-960d369ca5d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:57.560684 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc3124-1193-4125-87e8-960d369ca5d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:57.560949 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc3124-1193-4125-87e8-960d369ca5d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:57.561217 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc3124-1193-4125-87e8-960d369ca5d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:57.570319 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efc3124-1193-4125-87e8-960d369ca5d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:57.571734 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efc3124-1193-4125-87e8-960d369ca5d5 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:57.581426 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1103/2190] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:43:57.890944 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaff712-a718-4cdb-8d3a-a9fda632a386 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:57.892502 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ddaff712-a718-4cdb-8d3a-a9fda632a386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:57.911514 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaff712-a718-4cdb-8d3a-a9fda632a386 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:57.917288 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c001e4d-c875-4332-b431-fd294944ffdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:57.918032 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c001e4d-c875-4332-b431-fd294944ffdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:57.918363 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c001e4d-c875-4332-b431-fd294944ffdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:57.918648 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c001e4d-c875-4332-b431-fd294944ffdb admin admin] PATH_INFO: `/v3/projects/97672b3a5d044644961c1844b3c17357` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:57.921204 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c001e4d-c875-4332-b431-fd294944ffdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=97672b3a5d044644961c1844b3c17357)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:57.923243 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c001e4d-c875-4332-b431-fd294944ffdb admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:57.972075 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4c001e4d-c875-4332-b431-fd294944ffdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97672b3a5d044644961c1844b3c17357', 'request_id': 'req-4c001e4d-c875-4332-b431-fd294944ffdb'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:57.980708 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4c001e4d-c875-4332-b431-fd294944ffdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97672b3a5d044644961c1844b3c17357', 'request_id': 'req-4c001e4d-c875-4332-b431-fd294944ffdb'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:57.987588 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4c001e4d-c875-4332-b431-fd294944ffdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97672b3a5d044644961c1844b3c17357', 'request_id': 'req-4c001e4d-c875-4332-b431-fd294944ffdb'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:57.998172 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4c001e4d-c875-4332-b431-fd294944ffdb admin admin] The token cache is being invalidate because project 97672b3a5d044644961c1844b3c17357 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:43:57.998456 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4c001e4d-c875-4332-b431-fd294944ffdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:57.999097 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-4c001e4d-c875-4332-b431-fd294944ffdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:58.000772 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1088/2191] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:43:57 2022] DELETE /identity/v3/projects/97672b3a5d044644961c1844b3c17357 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:58.004533 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc3124-1193-4125-87e8-960d369ca5d5 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:58.014412 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc3124-1193-4125-87e8-960d369ca5d5 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:58.072259 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e24ff8-1c08-4a7b-9590-5dd5c4893f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:58.072920 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e24ff8-1c08-4a7b-9590-5dd5c4893f6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:58.073196 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e24ff8-1c08-4a7b-9590-5dd5c4893f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:58.073465 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e24ff8-1c08-4a7b-9590-5dd5c4893f6c admin admin] PATH_INFO: `/v3/users/ba26cb0db94143c4811889b80b5b54f5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:58.074645 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e24ff8-1c08-4a7b-9590-5dd5c4893f6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba26cb0db94143c4811889b80b5b54f5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:58.076581 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e24ff8-1c08-4a7b-9590-5dd5c4893f6c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:58.125374 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b7e24ff8-1c08-4a7b-9590-5dd5c4893f6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba26cb0db94143c4811889b80b5b54f5'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:58.126093 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b7e24ff8-1c08-4a7b-9590-5dd5c4893f6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba26cb0db94143c4811889b80b5b54f5'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:58.137888 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b7e24ff8-1c08-4a7b-9590-5dd5c4893f6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba26cb0db94143c4811889b80b5b54f5'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:58.146389 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b7e24ff8-1c08-4a7b-9590-5dd5c4893f6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba26cb0db94143c4811889b80b5b54f5'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:58.154694 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b7e24ff8-1c08-4a7b-9590-5dd5c4893f6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba26cb0db94143c4811889b80b5b54f5'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:58.161294 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1104/2192] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:43:58 2022] DELETE /identity/v3/users/ba26cb0db94143c4811889b80b5b54f5 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:58.504036 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c001e4d-c875-4332-b431-fd294944ffdb admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:58.505558 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4c001e4d-c875-4332-b431-fd294944ffdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:58.529233 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c001e4d-c875-4332-b431-fd294944ffdb admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:58.597206 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5810c38e-d1b2-4aab-b249-0850282c3f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:58.597837 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5810c38e-d1b2-4aab-b249-0850282c3f31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:58.598143 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5810c38e-d1b2-4aab-b249-0850282c3f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:58.598438 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5810c38e-d1b2-4aab-b249-0850282c3f31 admin admin] PATH_INFO: `/v3/projects/9f15387d660347b1805a900418d4a6fb` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:58.599544 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5810c38e-d1b2-4aab-b249-0850282c3f31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f15387d660347b1805a900418d4a6fb)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:58.601466 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5810c38e-d1b2-4aab-b249-0850282c3f31 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:58.637723 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:58.649496 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5810c38e-d1b2-4aab-b249-0850282c3f31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f15387d660347b1805a900418d4a6fb', 'request_id': 'req-5810c38e-d1b2-4aab-b249-0850282c3f31'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:58.656066 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5810c38e-d1b2-4aab-b249-0850282c3f31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f15387d660347b1805a900418d4a6fb', 'request_id': 'req-5810c38e-d1b2-4aab-b249-0850282c3f31'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:58.662334 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5810c38e-d1b2-4aab-b249-0850282c3f31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f15387d660347b1805a900418d4a6fb', 'request_id': 'req-5810c38e-d1b2-4aab-b249-0850282c3f31'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:58.672840 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5810c38e-d1b2-4aab-b249-0850282c3f31 admin admin] The token cache is being invalidate because project 9f15387d660347b1805a900418d4a6fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:43:58.673186 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5810c38e-d1b2-4aab-b249-0850282c3f31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:58.673965 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5810c38e-d1b2-4aab-b249-0850282c3f31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:58.675487 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1089/2193] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:43:58 2022] DELETE /identity/v3/projects/9f15387d660347b1805a900418d4a6fb => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:58.680841 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176748da-5b9e-46bc-9847-e93c6c58d138 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:58.681303 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176748da-5b9e-46bc-9847-e93c6c58d138 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:58.681494 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176748da-5b9e-46bc-9847-e93c6c58d138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:58.701499 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-176748da-5b9e-46bc-9847-e93c6c58d138 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:58.702724 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-176748da-5b9e-46bc-9847-e93c6c58d138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:58.732674 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:58.766275 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1105/2194] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:43:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:43:58.773671 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27df511-004f-4512-8167-5311af158667 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:58.774050 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27df511-004f-4512-8167-5311af158667 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:58.774429 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27df511-004f-4512-8167-5311af158667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:58.799505 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-a27df511-004f-4512-8167-5311af158667 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:58.800718 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a27df511-004f-4512-8167-5311af158667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:58.809122 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1090/2195] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:43:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:58.813555 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176748da-5b9e-46bc-9847-e93c6c58d138 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:58.824003 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176748da-5b9e-46bc-9847-e93c6c58d138 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:58.828888 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de850d67-d474-44ba-b883-c04955bceaf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:58.829516 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de850d67-d474-44ba-b883-c04955bceaf1 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:58.830211 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de850d67-d474-44ba-b883-c04955bceaf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:58.830211 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de850d67-d474-44ba-b883-c04955bceaf1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:58.830722 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de850d67-d474-44ba-b883-c04955bceaf1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:43:58.831029 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de850d67-d474-44ba-b883-c04955bceaf1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:58.832665 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de850d67-d474-44ba-b883-c04955bceaf1 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:58.846757 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1106/2196] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:43:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:43:58.851199 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27df511-004f-4512-8167-5311af158667 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:58.861199 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27df511-004f-4512-8167-5311af158667 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:58.866808 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719bfb82-4f4f-4185-a3f3-56baaaf55dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:58.867427 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bfb82-4f4f-4185-a3f3-56baaaf55dae admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:58.867704 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bfb82-4f4f-4185-a3f3-56baaaf55dae admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:58.867973 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719bfb82-4f4f-4185-a3f3-56baaaf55dae admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:58.868612 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719bfb82-4f4f-4185-a3f3-56baaaf55dae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:58.870732 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719bfb82-4f4f-4185-a3f3-56baaaf55dae admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:58.909278 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1091/2197] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:43:58 2022] POST /identity/v3/projects => generated 359 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:58.910518 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de850d67-d474-44ba-b883-c04955bceaf1 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:58.920914 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de850d67-d474-44ba-b883-c04955bceaf1 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:58.987512 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3084b8b6-9407-4b76-b51f-275d2a2ef263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:58.988260 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3084b8b6-9407-4b76-b51f-275d2a2ef263 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:58.988379 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3084b8b6-9407-4b76-b51f-275d2a2ef263 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:58.988649 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3084b8b6-9407-4b76-b51f-275d2a2ef263 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:58.989268 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3084b8b6-9407-4b76-b51f-275d2a2ef263 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:58.991968 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3084b8b6-9407-4b76-b51f-275d2a2ef263 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:59.014649 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1107/2198] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:43:58 2022] POST /identity/v3/users => generated 280 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:43:59.018681 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719bfb82-4f4f-4185-a3f3-56baaaf55dae admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:59.029643 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719bfb82-4f4f-4185-a3f3-56baaaf55dae admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:59.037377 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37245356-9490-45e1-ba4d-5fa29e64aa8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:59.038645 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37245356-9490-45e1-ba4d-5fa29e64aa8e admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:59.038925 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37245356-9490-45e1-ba4d-5fa29e64aa8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:59.039245 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37245356-9490-45e1-ba4d-5fa29e64aa8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:59.039951 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37245356-9490-45e1-ba4d-5fa29e64aa8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:59.041687 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37245356-9490-45e1-ba4d-5fa29e64aa8e admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:59.057140 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1092/2199] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:59.059400 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3084b8b6-9407-4b76-b51f-275d2a2ef263 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:59.070103 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3084b8b6-9407-4b76-b51f-275d2a2ef263 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:59.076046 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f87b99-3714-4999-905a-8e92eeb8fdcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:59.077206 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f87b99-3714-4999-905a-8e92eeb8fdcc admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:59.077206 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f87b99-3714-4999-905a-8e92eeb8fdcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:59.077574 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f87b99-3714-4999-905a-8e92eeb8fdcc admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:59.078290 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f87b99-3714-4999-905a-8e92eeb8fdcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:59.080281 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f87b99-3714-4999-905a-8e92eeb8fdcc admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:59.095097 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1108/2200] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:43:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:43:59.097491 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37245356-9490-45e1-ba4d-5fa29e64aa8e admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:59.107372 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37245356-9490-45e1-ba4d-5fa29e64aa8e admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:59.112160 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c23fe-df65-42d1-926f-49fc86f0845d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:59.112991 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c23fe-df65-42d1-926f-49fc86f0845d admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:59.113332 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c23fe-df65-42d1-926f-49fc86f0845d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:59.113689 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c23fe-df65-42d1-926f-49fc86f0845d admin admin] PATH_INFO: `/v3/projects/d6ece3e0b896496d9e1004d384815e39/users/c458d35815824dc8b1b07045e4202d3f/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:59.132074 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83c23fe-df65-42d1-926f-49fc86f0845d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6ece3e0b896496d9e1004d384815e39, user_id=c458d35815824dc8b1b07045e4202d3f, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:59.135486 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83c23fe-df65-42d1-926f-49fc86f0845d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:59.152001 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1093/2201] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:43:59 2022] PUT /identity/v3/projects/d6ece3e0b896496d9e1004d384815e39/users/c458d35815824dc8b1b07045e4202d3f/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:59.164027 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e480f-a8c1-4acb-8345-d45edffaa63b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:59.164311 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e480f-a8c1-4acb-8345-d45edffaa63b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:59.164582 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e480f-a8c1-4acb-8345-d45edffaa63b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:59.194410 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-c38e480f-a8c1-4acb-8345-d45edffaa63b None None] MFA Rules not processed for user `c458d35815824dc8b1b07045e4202d3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:59.195638 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c38e480f-a8c1-4acb-8345-d45edffaa63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:59.284746 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1109/2202] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:43:59 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:43:59.320965 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c23fe-df65-42d1-926f-49fc86f0845d admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:59.322701 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a83c23fe-df65-42d1-926f-49fc86f0845d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:59.381394 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c23fe-df65-42d1-926f-49fc86f0845d admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:59.386268 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e571163-8c90-4c8c-b559-8d6fb84299b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:59.386893 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e571163-8c90-4c8c-b559-8d6fb84299b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:59.387167 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e571163-8c90-4c8c-b559-8d6fb84299b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:59.387435 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e571163-8c90-4c8c-b559-8d6fb84299b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:59.396340 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e571163-8c90-4c8c-b559-8d6fb84299b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:59.399665 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e571163-8c90-4c8c-b559-8d6fb84299b4 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:59.413606 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1094/2203] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:59 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:43:59.429205 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38e480f-a8c1-4acb-8345-d45edffaa63b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:59.462072 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:43:59.494417 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38e480f-a8c1-4acb-8345-d45edffaa63b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:59.499298 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fa84dd-301d-448e-8ddb-0c774da0b5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:59.499926 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa84dd-301d-448e-8ddb-0c774da0b5a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:59.500327 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa84dd-301d-448e-8ddb-0c774da0b5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:59.500605 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa84dd-301d-448e-8ddb-0c774da0b5a2 admin admin] PATH_INFO: `/v3/users/c458d35815824dc8b1b07045e4202d3f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:59.501731 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fa84dd-301d-448e-8ddb-0c774da0b5a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c458d35815824dc8b1b07045e4202d3f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:59.504328 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fa84dd-301d-448e-8ddb-0c774da0b5a2 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:59.575619 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-73fa84dd-301d-448e-8ddb-0c774da0b5a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c458d35815824dc8b1b07045e4202d3f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:59.576564 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-73fa84dd-301d-448e-8ddb-0c774da0b5a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c458d35815824dc8b1b07045e4202d3f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:59.594503 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-73fa84dd-301d-448e-8ddb-0c774da0b5a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c458d35815824dc8b1b07045e4202d3f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:59.604154 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-73fa84dd-301d-448e-8ddb-0c774da0b5a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c458d35815824dc8b1b07045e4202d3f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:59.616374 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-73fa84dd-301d-448e-8ddb-0c774da0b5a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c458d35815824dc8b1b07045e4202d3f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:43:59.625237 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1110/2204] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:43:59 2022] DELETE /identity/v3/users/c458d35815824dc8b1b07045e4202d3f => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:43:59.630743 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf73bd-5242-4b8b-9f0f-9718695a0766 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:59.631246 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf73bd-5242-4b8b-9f0f-9718695a0766 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:59.631620 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf73bd-5242-4b8b-9f0f-9718695a0766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:59.654004 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-c2cf73bd-5242-4b8b-9f0f-9718695a0766 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:43:59.655317 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c2cf73bd-5242-4b8b-9f0f-9718695a0766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:59.738516 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1095/2205] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:43:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:43:59.753345 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fa84dd-301d-448e-8ddb-0c774da0b5a2 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:43:59.755398 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-73fa84dd-301d-448e-8ddb-0c774da0b5a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:43:59.826030 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fa84dd-301d-448e-8ddb-0c774da0b5a2 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:43:59.826891 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06955f23-e092-4796-8a5d-28e4128dc2c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:43:59.827575 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06955f23-e092-4796-8a5d-28e4128dc2c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:43:59.827933 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06955f23-e092-4796-8a5d-28e4128dc2c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:43:59.828294 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06955f23-e092-4796-8a5d-28e4128dc2c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:43:59.838375 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06955f23-e092-4796-8a5d-28e4128dc2c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:43:59.839808 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06955f23-e092-4796-8a5d-28e4128dc2c7 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:43:59.850122 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1111/2206] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:43:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:44:00.307138 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cf73bd-5242-4b8b-9f0f-9718695a0766 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:00.308656 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c2cf73bd-5242-4b8b-9f0f-9718695a0766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:00.315938 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:00.322799 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cf73bd-5242-4b8b-9f0f-9718695a0766 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:00.329213 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe6cd7c-4298-45c4-8047-35239568f0bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:00.331038 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe6cd7c-4298-45c4-8047-35239568f0bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:00.331425 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe6cd7c-4298-45c4-8047-35239568f0bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:00.332057 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe6cd7c-4298-45c4-8047-35239568f0bb admin admin] PATH_INFO: `/v3/projects/d6ece3e0b896496d9e1004d384815e39` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:00.333567 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe6cd7c-4298-45c4-8047-35239568f0bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6ece3e0b896496d9e1004d384815e39)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:00.336399 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe6cd7c-4298-45c4-8047-35239568f0bb admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:00.408887 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f036a8-d840-40d0-9c0e-1452aea922fe None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:00.409174 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f036a8-d840-40d0-9c0e-1452aea922fe None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:00.409451 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f036a8-d840-40d0-9c0e-1452aea922fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:00.426158 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ffe6cd7c-4298-45c4-8047-35239568f0bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6ece3e0b896496d9e1004d384815e39', 'request_id': 'req-ffe6cd7c-4298-45c4-8047-35239568f0bb'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:00.441666 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-c9f036a8-d840-40d0-9c0e-1452aea922fe None None] MFA Rules not processed for user `36515ae6f91748babdcf6a9a72f59ed1`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:00.443027 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c9f036a8-d840-40d0-9c0e-1452aea922fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:00.447945 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ffe6cd7c-4298-45c4-8047-35239568f0bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6ece3e0b896496d9e1004d384815e39', 'request_id': 'req-ffe6cd7c-4298-45c4-8047-35239568f0bb'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:00.459189 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ffe6cd7c-4298-45c4-8047-35239568f0bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6ece3e0b896496d9e1004d384815e39', 'request_id': 'req-ffe6cd7c-4298-45c4-8047-35239568f0bb'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:00.475889 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ffe6cd7c-4298-45c4-8047-35239568f0bb admin admin] The token cache is being invalidate because project d6ece3e0b896496d9e1004d384815e39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:44:00.476789 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ffe6cd7c-4298-45c4-8047-35239568f0bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:00.477466 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ffe6cd7c-4298-45c4-8047-35239568f0bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:00.483707 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1096/2207] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:44:00 2022] DELETE /identity/v3/projects/d6ece3e0b896496d9e1004d384815e39 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:00.485370 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c6933-62a3-46ee-b7d0-d224e53d2bb0 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:00.485654 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c6933-62a3-46ee-b7d0-d224e53d2bb0 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:00.485953 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c6933-62a3-46ee-b7d0-d224e53d2bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:00.500994 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:00.519724 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1112/2208] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:44:00 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:00.519919 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-af2c6933-62a3-46ee-b7d0-d224e53d2bb0 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:00.519919 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-af2c6933-62a3-46ee-b7d0-d224e53d2bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:00.583374 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1097/2209] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:44:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:00.591021 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c20b72-6a0a-45e8-a5e9-1d5987229fd3 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:00.591308 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c20b72-6a0a-45e8-a5e9-1d5987229fd3 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:00.591611 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c20b72-6a0a-45e8-a5e9-1d5987229fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:00.611443 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-e6c20b72-6a0a-45e8-a5e9-1d5987229fd3 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:00.612642 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e6c20b72-6a0a-45e8-a5e9-1d5987229fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:00.621140 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1113/2210] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:44:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:00.625622 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c6933-62a3-46ee-b7d0-d224e53d2bb0 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:00.635345 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c6933-62a3-46ee-b7d0-d224e53d2bb0 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:00.640174 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5286c390-ccbc-4d55-b430-b63aef7373e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:00.640769 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5286c390-ccbc-4d55-b430-b63aef7373e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:00.641040 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5286c390-ccbc-4d55-b430-b63aef7373e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:00.641304 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5286c390-ccbc-4d55-b430-b63aef7373e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:00.641970 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5286c390-ccbc-4d55-b430-b63aef7373e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:44:00.642270 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5286c390-ccbc-4d55-b430-b63aef7373e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:00.643852 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5286c390-ccbc-4d55-b430-b63aef7373e3 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:00.656593 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1098/2211] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:44:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:44:00.660525 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c20b72-6a0a-45e8-a5e9-1d5987229fd3 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:00.670763 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c20b72-6a0a-45e8-a5e9-1d5987229fd3 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:00.675310 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe94ca-87fd-4d28-bf57-e89ba49d0a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:00.675899 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe94ca-87fd-4d28-bf57-e89ba49d0a6d admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:00.676175 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe94ca-87fd-4d28-bf57-e89ba49d0a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:00.676450 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe94ca-87fd-4d28-bf57-e89ba49d0a6d admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:00.677062 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebe94ca-87fd-4d28-bf57-e89ba49d0a6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:00.679665 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebe94ca-87fd-4d28-bf57-e89ba49d0a6d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:00.706199 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1114/2212] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:44:00 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:44:00.710219 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5286c390-ccbc-4d55-b430-b63aef7373e3 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:00.719974 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5286c390-ccbc-4d55-b430-b63aef7373e3 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:00.781490 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935c0cf7-8586-4557-b210-41fd475ed64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:00.782170 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c0cf7-8586-4557-b210-41fd475ed64a admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:00.782452 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c0cf7-8586-4557-b210-41fd475ed64a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:00.782723 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c0cf7-8586-4557-b210-41fd475ed64a admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:00.783363 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935c0cf7-8586-4557-b210-41fd475ed64a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:00.785298 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935c0cf7-8586-4557-b210-41fd475ed64a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:00.810750 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1099/2213] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:44:00 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:44:00.815021 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe94ca-87fd-4d28-bf57-e89ba49d0a6d admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:00.826831 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe94ca-87fd-4d28-bf57-e89ba49d0a6d admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:00.833443 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca73e97-b72d-4bf8-b335-5c924d8dfef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:00.834112 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca73e97-b72d-4bf8-b335-5c924d8dfef1 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:00.834395 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca73e97-b72d-4bf8-b335-5c924d8dfef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:00.834674 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca73e97-b72d-4bf8-b335-5c924d8dfef1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:00.835371 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca73e97-b72d-4bf8-b335-5c924d8dfef1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:00.837012 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca73e97-b72d-4bf8-b335-5c924d8dfef1 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:00.848284 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1115/2214] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:44:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:44:00.852726 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935c0cf7-8586-4557-b210-41fd475ed64a admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:00.859453 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:00.867410 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935c0cf7-8586-4557-b210-41fd475ed64a admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:00.873802 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e5574f-1ff8-4aa0-8a27-15c8475bcc63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:00.874611 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5574f-1ff8-4aa0-8a27-15c8475bcc63 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:00.874990 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5574f-1ff8-4aa0-8a27-15c8475bcc63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:00.875361 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5574f-1ff8-4aa0-8a27-15c8475bcc63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:00.876125 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e5574f-1ff8-4aa0-8a27-15c8475bcc63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:00.877842 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e5574f-1ff8-4aa0-8a27-15c8475bcc63 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:00.889572 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1100/2215] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:44:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:44:00.893990 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca73e97-b72d-4bf8-b335-5c924d8dfef1 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:00.904439 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca73e97-b72d-4bf8-b335-5c924d8dfef1 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:00.909329 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b2796-994b-4fcb-aee9-290bce60300d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:00.910104 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b2796-994b-4fcb-aee9-290bce60300d admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:00.910387 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b2796-994b-4fcb-aee9-290bce60300d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:00.910662 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b2796-994b-4fcb-aee9-290bce60300d admin admin] PATH_INFO: `/v3/projects/9fdfbb13c5a34b72b1bb3646a731a3c8/users/d74fb7cfd3004ea0a839bd1af00f6b11/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:00.928208 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84b2796-994b-4fcb-aee9-290bce60300d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fdfbb13c5a34b72b1bb3646a731a3c8, user_id=d74fb7cfd3004ea0a839bd1af00f6b11, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:00.931764 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84b2796-994b-4fcb-aee9-290bce60300d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:00.949963 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1116/2216] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:44:00 2022] PUT /identity/v3/projects/9fdfbb13c5a34b72b1bb3646a731a3c8/users/d74fb7cfd3004ea0a839bd1af00f6b11/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:00.957643 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b785eb-13f3-4ffa-91cf-6464668d3e66 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:00.957952 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b785eb-13f3-4ffa-91cf-6464668d3e66 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:00.958329 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b785eb-13f3-4ffa-91cf-6464668d3e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:00.987726 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-19b785eb-13f3-4ffa-91cf-6464668d3e66 None None] MFA Rules not processed for user `d74fb7cfd3004ea0a839bd1af00f6b11`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:00.989047 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-19b785eb-13f3-4ffa-91cf-6464668d3e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:01.059720 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1101/2217] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:44:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:01.071602 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b2796-994b-4fcb-aee9-290bce60300d admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:01.073333 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a84b2796-994b-4fcb-aee9-290bce60300d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:01.134723 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b2796-994b-4fcb-aee9-290bce60300d admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:01.138654 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed7080d-bba8-4935-b20e-a24eb1166dca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:01.139374 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7080d-bba8-4935-b20e-a24eb1166dca service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:01.139705 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7080d-bba8-4935-b20e-a24eb1166dca service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:01.140051 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7080d-bba8-4935-b20e-a24eb1166dca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:01.149221 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed7080d-bba8-4935-b20e-a24eb1166dca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:01.151256 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed7080d-bba8-4935-b20e-a24eb1166dca service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:01.160131 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1117/2218] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:44:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:44:02.132754 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de5880b-f417-44ea-808d-5f832af11a7b None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:02.133608 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de5880b-f417-44ea-808d-5f832af11a7b None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:02.134532 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de5880b-f417-44ea-808d-5f832af11a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:02.181185 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-4de5880b-f417-44ea-808d-5f832af11a7b None None] MFA Rules not processed for user `c81e5fb007614cf2992f5e02e85d6307`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:02.183027 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4de5880b-f417-44ea-808d-5f832af11a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:02.260180 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1102/2219] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Sep 28 15:44:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:02.278776 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed7080d-bba8-4935-b20e-a24eb1166dca service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:02.280273 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-eed7080d-bba8-4935-b20e-a24eb1166dca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:02.380801 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed7080d-bba8-4935-b20e-a24eb1166dca service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:02.384742 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d141472-a28b-4c04-95f8-622f7fc810d6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '328051fe7607464f8f5dc0dcc01a817a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:02.385389 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d141472-a28b-4c04-95f8-622f7fc810d6 service swift] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:02.385948 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d141472-a28b-4c04-95f8-622f7fc810d6 service swift] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:02.386320 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d141472-a28b-4c04-95f8-622f7fc810d6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:02.394778 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d141472-a28b-4c04-95f8-622f7fc810d6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:02.396346 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d141472-a28b-4c04-95f8-622f7fc810d6 service swift] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:02.404576 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1118/2220] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 15:44:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 15:44:11.154341 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de5880b-f417-44ea-808d-5f832af11a7b None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:11.156376 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4de5880b-f417-44ea-808d-5f832af11a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:11.275555 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de5880b-f417-44ea-808d-5f832af11a7b None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:11.281602 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2814d125-9cb5-4615-873a-e106c7279f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:11.283046 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2814d125-9cb5-4615-873a-e106c7279f22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:11.283663 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2814d125-9cb5-4615-873a-e106c7279f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:11.284005 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2814d125-9cb5-4615-873a-e106c7279f22 admin admin] PATH_INFO: `/v3/users/571e878f30854e2bbf6132d96fd98e8a` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:11.285422 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2814d125-9cb5-4615-873a-e106c7279f22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=571e878f30854e2bbf6132d96fd98e8a)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:11.288909 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2814d125-9cb5-4615-873a-e106c7279f22 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:11.346593 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:11.383617 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-2814d125-9cb5-4615-873a-e106c7279f22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '571e878f30854e2bbf6132d96fd98e8a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:11.392921 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-2814d125-9cb5-4615-873a-e106c7279f22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '571e878f30854e2bbf6132d96fd98e8a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:11.394044 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-2814d125-9cb5-4615-873a-e106c7279f22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '571e878f30854e2bbf6132d96fd98e8a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:11.405270 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-2814d125-9cb5-4615-873a-e106c7279f22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '571e878f30854e2bbf6132d96fd98e8a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:11.406385 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-2814d125-9cb5-4615-873a-e106c7279f22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '571e878f30854e2bbf6132d96fd98e8a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:11.488584 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1103/2221] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:44:11 2022] DELETE /identity/v3/users/571e878f30854e2bbf6132d96fd98e8a => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:44:11.489227 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27576a9-a143-4cfc-9f6d-877991bd31b1 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:11.490409 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27576a9-a143-4cfc-9f6d-877991bd31b1 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:11.490911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27576a9-a143-4cfc-9f6d-877991bd31b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:11.525738 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d27576a9-a143-4cfc-9f6d-877991bd31b1 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:11.526954 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d27576a9-a143-4cfc-9f6d-877991bd31b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:11.555863 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:11.702501 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2814d125-9cb5-4615-873a-e106c7279f22 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:11.704064 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-2814d125-9cb5-4615-873a-e106c7279f22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:11.711938 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1119/2222] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:44:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:11.823308 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2814d125-9cb5-4615-873a-e106c7279f22 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:11.828983 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6542b53e-05b0-4be2-a266-c3c0ba4d960e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:11.829606 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6542b53e-05b0-4be2-a266-c3c0ba4d960e service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:11.829899 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6542b53e-05b0-4be2-a266-c3c0ba4d960e service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:11.831330 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6542b53e-05b0-4be2-a266-c3c0ba4d960e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:11.857753 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6542b53e-05b0-4be2-a266-c3c0ba4d960e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:11.861219 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6542b53e-05b0-4be2-a266-c3c0ba4d960e service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:11.881704 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1104/2223] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:44:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:44:13.266263 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27576a9-a143-4cfc-9f6d-877991bd31b1 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:13.266263 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d27576a9-a143-4cfc-9f6d-877991bd31b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:13.311793 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27576a9-a143-4cfc-9f6d-877991bd31b1 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:13.329068 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5dc517-1aa7-4e70-b860-15b0d5a13573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:13.330088 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5dc517-1aa7-4e70-b860-15b0d5a13573 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:13.330609 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5dc517-1aa7-4e70-b860-15b0d5a13573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:13.331032 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5dc517-1aa7-4e70-b860-15b0d5a13573 admin admin] PATH_INFO: `/v3/projects/948574e1f3664ae7a78e706c6f780e62` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:13.336348 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5dc517-1aa7-4e70-b860-15b0d5a13573 admin admin] RBAC: Authorizing `identity:delete_project(project_id=948574e1f3664ae7a78e706c6f780e62)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:13.340486 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5dc517-1aa7-4e70-b860-15b0d5a13573 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:13.455409 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ee5dc517-1aa7-4e70-b860-15b0d5a13573 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '948574e1f3664ae7a78e706c6f780e62', 'request_id': 'req-ee5dc517-1aa7-4e70-b860-15b0d5a13573'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:13.472073 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ee5dc517-1aa7-4e70-b860-15b0d5a13573 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '948574e1f3664ae7a78e706c6f780e62', 'request_id': 'req-ee5dc517-1aa7-4e70-b860-15b0d5a13573'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:13.483400 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ee5dc517-1aa7-4e70-b860-15b0d5a13573 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '948574e1f3664ae7a78e706c6f780e62', 'request_id': 'req-ee5dc517-1aa7-4e70-b860-15b0d5a13573'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:13.494887 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ee5dc517-1aa7-4e70-b860-15b0d5a13573 admin admin] The token cache is being invalidate because project 948574e1f3664ae7a78e706c6f780e62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:44:13.495432 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ee5dc517-1aa7-4e70-b860-15b0d5a13573 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:13.496073 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ee5dc517-1aa7-4e70-b860-15b0d5a13573 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:13.498116 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1120/2224] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:44:13 2022] DELETE /identity/v3/projects/948574e1f3664ae7a78e706c6f780e62 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:13.502621 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d41923-9aaf-4948-a43e-37b2c8783619 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:13.503620 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d41923-9aaf-4948-a43e-37b2c8783619 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:13.504078 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d41923-9aaf-4948-a43e-37b2c8783619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:13.526849 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-56d41923-9aaf-4948-a43e-37b2c8783619 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:13.528156 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-56d41923-9aaf-4948-a43e-37b2c8783619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:13.591883 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1105/2225] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:44:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:13.600234 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006b3c1-c412-4162-b051-1cf85eb05bb7 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:13.600633 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006b3c1-c412-4162-b051-1cf85eb05bb7 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:13.600998 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006b3c1-c412-4162-b051-1cf85eb05bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:13.621060 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-4006b3c1-c412-4162-b051-1cf85eb05bb7 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:13.622794 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4006b3c1-c412-4162-b051-1cf85eb05bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:13.632216 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1121/2226] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:44:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:44:13.637225 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d41923-9aaf-4948-a43e-37b2c8783619 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:13.649221 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d41923-9aaf-4948-a43e-37b2c8783619 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:13.654724 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5715729-a5b5-4ddc-9fbc-9eed2a3d4b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:13.655501 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5715729-a5b5-4ddc-9fbc-9eed2a3d4b55 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:13.655949 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5715729-a5b5-4ddc-9fbc-9eed2a3d4b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:13.656460 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5715729-a5b5-4ddc-9fbc-9eed2a3d4b55 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:13.657305 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5715729-a5b5-4ddc-9fbc-9eed2a3d4b55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:44:13.657900 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5715729-a5b5-4ddc-9fbc-9eed2a3d4b55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:13.659724 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5715729-a5b5-4ddc-9fbc-9eed2a3d4b55 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:13.677197 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1106/2227] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:44:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:44:13.678422 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006b3c1-c412-4162-b051-1cf85eb05bb7 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:13.688321 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006b3c1-c412-4162-b051-1cf85eb05bb7 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:13.692941 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7cd365-743c-495b-8607-400a9e809338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:13.694338 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7cd365-743c-495b-8607-400a9e809338 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:13.694764 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7cd365-743c-495b-8607-400a9e809338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:13.695202 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7cd365-743c-495b-8607-400a9e809338 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:13.695827 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7cd365-743c-495b-8607-400a9e809338 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:13.697811 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7cd365-743c-495b-8607-400a9e809338 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:13.726620 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1122/2228] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:44:13 2022] POST /identity/v3/projects => generated 363 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:44:13.737246 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5715729-a5b5-4ddc-9fbc-9eed2a3d4b55 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:13.748436 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5715729-a5b5-4ddc-9fbc-9eed2a3d4b55 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:13.780892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:13.809255 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a0603-4a21-4a16-808c-57f18809980a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:13.809889 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a0603-4a21-4a16-808c-57f18809980a admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:13.810175 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a0603-4a21-4a16-808c-57f18809980a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:13.810464 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a0603-4a21-4a16-808c-57f18809980a admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:13.811106 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1a0603-4a21-4a16-808c-57f18809980a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:13.813008 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1a0603-4a21-4a16-808c-57f18809980a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:13.838351 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1107/2229] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:44:13 2022] POST /identity/v3/users => generated 282 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:44:13.839972 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7cd365-743c-495b-8607-400a9e809338 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:13.850632 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7cd365-743c-495b-8607-400a9e809338 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:13.855402 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aaa32a-c9b9-4e19-9dda-7d553562719f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:13.856008 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aaa32a-c9b9-4e19-9dda-7d553562719f admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:13.856305 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aaa32a-c9b9-4e19-9dda-7d553562719f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:13.856566 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aaa32a-c9b9-4e19-9dda-7d553562719f admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:13.857205 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aaa32a-c9b9-4e19-9dda-7d553562719f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:13.858813 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aaa32a-c9b9-4e19-9dda-7d553562719f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:13.880029 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1123/2230] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:44:13 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:44:13.884739 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a0603-4a21-4a16-808c-57f18809980a admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:13.900321 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a0603-4a21-4a16-808c-57f18809980a admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:13.911404 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f511d-7436-4c35-a3d8-c1d754c9d508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:13.911962 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f511d-7436-4c35-a3d8-c1d754c9d508 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:13.913912 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f511d-7436-4c35-a3d8-c1d754c9d508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:13.914269 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f511d-7436-4c35-a3d8-c1d754c9d508 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:13.914938 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2f511d-7436-4c35-a3d8-c1d754c9d508 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:13.918206 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2f511d-7436-4c35-a3d8-c1d754c9d508 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:13.939954 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1108/2231] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:44:13 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:44:13.940828 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aaa32a-c9b9-4e19-9dda-7d553562719f admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:13.956853 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aaa32a-c9b9-4e19-9dda-7d553562719f admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:13.968738 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a26042-cc82-4ef3-ab9b-0929a7de0d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:13.969445 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a26042-cc82-4ef3-ab9b-0929a7de0d05 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:13.969726 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a26042-cc82-4ef3-ab9b-0929a7de0d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:13.970025 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a26042-cc82-4ef3-ab9b-0929a7de0d05 admin admin] PATH_INFO: `/v3/projects/e4267acfa5ad411c9b132c49946b67fd/users/bd7995d56920468c8da40fb74e5e14e7/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:13.995263 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a26042-cc82-4ef3-ab9b-0929a7de0d05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4267acfa5ad411c9b132c49946b67fd, user_id=bd7995d56920468c8da40fb74e5e14e7, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:13.998667 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a26042-cc82-4ef3-ab9b-0929a7de0d05 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:14.016953 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1124/2232] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:44:13 2022] PUT /identity/v3/projects/e4267acfa5ad411c9b132c49946b67fd/users/bd7995d56920468c8da40fb74e5e14e7/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:44:14.027532 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fba7c08-b5fd-41ca-b16e-0963ca8bc0d2 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:14.027816 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fba7c08-b5fd-41ca-b16e-0963ca8bc0d2 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:14.028090 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fba7c08-b5fd-41ca-b16e-0963ca8bc0d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:14.066789 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-2fba7c08-b5fd-41ca-b16e-0963ca8bc0d2 None None] MFA Rules not processed for user `bd7995d56920468c8da40fb74e5e14e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:14.068269 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-2fba7c08-b5fd-41ca-b16e-0963ca8bc0d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:14.142916 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1109/2233] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:44:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:14.154825 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a26042-cc82-4ef3-ab9b-0929a7de0d05 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:14.157420 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-00a26042-cc82-4ef3-ab9b-0929a7de0d05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:14.214299 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:14.228862 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a26042-cc82-4ef3-ab9b-0929a7de0d05 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:14.233635 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15af673-ab7f-4b9c-9ba7-1f5f306da0bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:14.234394 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15af673-ab7f-4b9c-9ba7-1f5f306da0bb service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:14.234742 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15af673-ab7f-4b9c-9ba7-1f5f306da0bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:14.235079 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15af673-ab7f-4b9c-9ba7-1f5f306da0bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:14.249402 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15af673-ab7f-4b9c-9ba7-1f5f306da0bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:14.251054 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15af673-ab7f-4b9c-9ba7-1f5f306da0bb service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:14.261032 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1125/2234] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:44:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:44:21.436186 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc0c600-9ddc-405f-9d1f-00298b41c836 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:21.436683 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc0c600-9ddc-405f-9d1f-00298b41c836 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:21.436768 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc0c600-9ddc-405f-9d1f-00298b41c836 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:21.438229 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1110/2235] 10.222.0.52 () {54 vars in 921 bytes} [Wed Sep 28 15:44:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:44:21.445130 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1675bb-5dcf-4a14-a0e1-72f5754754c2 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:21.445591 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1675bb-5dcf-4a14-a0e1-72f5754754c2 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:21.445700 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1675bb-5dcf-4a14-a0e1-72f5754754c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:21.465775 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-cd1675bb-5dcf-4a14-a0e1-72f5754754c2 None None] MFA Rules not processed for user `8daf14b64bf04b2ba74d2fe1fd41ff71`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:21.470301 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-cd1675bb-5dcf-4a14-a0e1-72f5754754c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:21.560858 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1126/2236] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Sep 28 15:44:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:21.578554 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc0c600-9ddc-405f-9d1f-00298b41c836 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:21.580833 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-bbc0c600-9ddc-405f-9d1f-00298b41c836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:21.641978 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc0c600-9ddc-405f-9d1f-00298b41c836 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:21.645479 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b31f54-b5d6-49d4-bb4a-59f09cfcb02a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6a46bbd249c4434ad7d9442754f098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:21.646105 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b31f54-b5d6-49d4-bb4a-59f09cfcb02a service glance] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:21.646383 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b31f54-b5d6-49d4-bb4a-59f09cfcb02a service glance] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:21.646654 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b31f54-b5d6-49d4-bb4a-59f09cfcb02a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:21.656607 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b31f54-b5d6-49d4-bb4a-59f09cfcb02a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:21.658046 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b31f54-b5d6-49d4-bb4a-59f09cfcb02a service glance] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:21.669133 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1111/2237] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:44:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:44:21.738807 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be902496-337d-4a1a-8b77-b16b5a560d00 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:21.739099 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be902496-337d-4a1a-8b77-b16b5a560d00 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:21.739409 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be902496-337d-4a1a-8b77-b16b5a560d00 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:21.741349 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1127/2238] 10.222.0.52 () {54 vars in 921 bytes} [Wed Sep 28 15:44:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:44:21.746075 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea616cd6-dabd-478c-bcc4-e18f109e1232 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:21.746375 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea616cd6-dabd-478c-bcc4-e18f109e1232 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:21.746632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea616cd6-dabd-478c-bcc4-e18f109e1232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:21.766355 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-ea616cd6-dabd-478c-bcc4-e18f109e1232 None None] MFA Rules not processed for user `b13b0533444a449ead00a389c5f6bd1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:21.767560 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ea616cd6-dabd-478c-bcc4-e18f109e1232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:21.810584 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:21.834004 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1112/2239] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Sep 28 15:44:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:21.853205 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be902496-337d-4a1a-8b77-b16b5a560d00 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:21.854998 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-be902496-337d-4a1a-8b77-b16b5a560d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:21.880903 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be902496-337d-4a1a-8b77-b16b5a560d00 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:21.886321 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee657af3-cf54-49fa-b192-7884fc63783e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:21.887188 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee657af3-cf54-49fa-b192-7884fc63783e service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:21.887555 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee657af3-cf54-49fa-b192-7884fc63783e service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:21.887867 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee657af3-cf54-49fa-b192-7884fc63783e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:21.896483 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee657af3-cf54-49fa-b192-7884fc63783e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:21.898792 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee657af3-cf54-49fa-b192-7884fc63783e service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:21.908247 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1128/2240] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:44:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:44:22.022423 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb84f1c-48fc-4622-892c-74766bd14c4f None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:22.022921 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb84f1c-48fc-4622-892c-74766bd14c4f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:22.023268 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb84f1c-48fc-4622-892c-74766bd14c4f None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:22.025151 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1113/2241] 10.222.0.52 () {54 vars in 921 bytes} [Wed Sep 28 15:44:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:44:22.032458 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e5eab1-a993-4d27-a216-491b03649f43 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:22.033138 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e5eab1-a993-4d27-a216-491b03649f43 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:22.033609 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e5eab1-a993-4d27-a216-491b03649f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:22.055328 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-16e5eab1-a993-4d27-a216-491b03649f43 None None] MFA Rules not processed for user `8daf14b64bf04b2ba74d2fe1fd41ff71`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:22.056682 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-16e5eab1-a993-4d27-a216-491b03649f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:22.069654 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1129/2242] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Sep 28 15:44:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:22.083288 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb84f1c-48fc-4622-892c-74766bd14c4f None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:22.085813 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ebb84f1c-48fc-4622-892c-74766bd14c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:22.103157 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb84f1c-48fc-4622-892c-74766bd14c4f None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:22.107302 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e96094-37ff-4ab3-aa61-4f31765628cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:22.107901 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e96094-37ff-4ab3-aa61-4f31765628cf service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:22.108371 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e96094-37ff-4ab3-aa61-4f31765628cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:22.108570 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e96094-37ff-4ab3-aa61-4f31765628cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:22.120033 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e96094-37ff-4ab3-aa61-4f31765628cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:22.121448 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e96094-37ff-4ab3-aa61-4f31765628cf service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:22.134535 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1114/2243] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:44:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:44:22.728551 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e5eab1-a993-4d27-a216-491b03649f43 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:22.730510 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-16e5eab1-a993-4d27-a216-491b03649f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:22.822808 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e5eab1-a993-4d27-a216-491b03649f43 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:22.827279 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567847b3-f3f6-414a-ab64-1cf8c59c2ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:22.827904 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567847b3-f3f6-414a-ab64-1cf8c59c2ef5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:22.828184 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567847b3-f3f6-414a-ab64-1cf8c59c2ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:22.828456 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567847b3-f3f6-414a-ab64-1cf8c59c2ef5 admin admin] PATH_INFO: `/v3/users/d74fb7cfd3004ea0a839bd1af00f6b11` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:22.829557 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567847b3-f3f6-414a-ab64-1cf8c59c2ef5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d74fb7cfd3004ea0a839bd1af00f6b11)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:22.831538 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567847b3-f3f6-414a-ab64-1cf8c59c2ef5 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:22.875748 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-567847b3-f3f6-414a-ab64-1cf8c59c2ef5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd74fb7cfd3004ea0a839bd1af00f6b11'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:22.876326 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-567847b3-f3f6-414a-ab64-1cf8c59c2ef5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd74fb7cfd3004ea0a839bd1af00f6b11'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:22.886888 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:22.889374 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-567847b3-f3f6-414a-ab64-1cf8c59c2ef5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd74fb7cfd3004ea0a839bd1af00f6b11'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:22.897890 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-567847b3-f3f6-414a-ab64-1cf8c59c2ef5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd74fb7cfd3004ea0a839bd1af00f6b11'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:22.905754 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-567847b3-f3f6-414a-ab64-1cf8c59c2ef5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd74fb7cfd3004ea0a839bd1af00f6b11'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:22.911638 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1130/2244] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:44:22 2022] DELETE /identity/v3/users/d74fb7cfd3004ea0a839bd1af00f6b11 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:22.916021 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec30ddd2-139b-4baa-bb26-06b1452a169c None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:22.916479 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec30ddd2-139b-4baa-bb26-06b1452a169c None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:22.916599 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec30ddd2-139b-4baa-bb26-06b1452a169c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:22.935551 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-ec30ddd2-139b-4baa-bb26-06b1452a169c None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:22.936715 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ec30ddd2-139b-4baa-bb26-06b1452a169c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:22.994543 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1115/2245] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:44:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:23.004308 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567847b3-f3f6-414a-ab64-1cf8c59c2ef5 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:23.005733 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-567847b3-f3f6-414a-ab64-1cf8c59c2ef5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:23.064397 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567847b3-f3f6-414a-ab64-1cf8c59c2ef5 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:23.068034 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f783a0-aa08-4684-a2a3-e48c6e021ba5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:23.068692 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f783a0-aa08-4684-a2a3-e48c6e021ba5 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:23.068969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f783a0-aa08-4684-a2a3-e48c6e021ba5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:23.069241 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f783a0-aa08-4684-a2a3-e48c6e021ba5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:23.078344 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f783a0-aa08-4684-a2a3-e48c6e021ba5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:23.079765 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f783a0-aa08-4684-a2a3-e48c6e021ba5 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:23.089139 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1131/2246] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:44:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:44:23.804037 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec30ddd2-139b-4baa-bb26-06b1452a169c None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:23.805713 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ec30ddd2-139b-4baa-bb26-06b1452a169c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:23.823546 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec30ddd2-139b-4baa-bb26-06b1452a169c None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:23.855995 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b1b6d3-3371-4707-abc7-32298ed95337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:23.856498 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b1b6d3-3371-4707-abc7-32298ed95337 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:23.856778 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b1b6d3-3371-4707-abc7-32298ed95337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:23.857049 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b1b6d3-3371-4707-abc7-32298ed95337 admin admin] PATH_INFO: `/v3/projects/9fdfbb13c5a34b72b1bb3646a731a3c8` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:23.858289 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b1b6d3-3371-4707-abc7-32298ed95337 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fdfbb13c5a34b72b1bb3646a731a3c8)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:23.861081 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b1b6d3-3371-4707-abc7-32298ed95337 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:23.935170 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-f2b1b6d3-3371-4707-abc7-32298ed95337 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fdfbb13c5a34b72b1bb3646a731a3c8', 'request_id': 'req-f2b1b6d3-3371-4707-abc7-32298ed95337'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:23.947431 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-f2b1b6d3-3371-4707-abc7-32298ed95337 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fdfbb13c5a34b72b1bb3646a731a3c8', 'request_id': 'req-f2b1b6d3-3371-4707-abc7-32298ed95337'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:23.955418 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-f2b1b6d3-3371-4707-abc7-32298ed95337 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fdfbb13c5a34b72b1bb3646a731a3c8', 'request_id': 'req-f2b1b6d3-3371-4707-abc7-32298ed95337'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:23.967343 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-f2b1b6d3-3371-4707-abc7-32298ed95337 admin admin] The token cache is being invalidate because project 9fdfbb13c5a34b72b1bb3646a731a3c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:44:23.967730 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-f2b1b6d3-3371-4707-abc7-32298ed95337 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:23.968464 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-f2b1b6d3-3371-4707-abc7-32298ed95337 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:23.970053 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1116/2247] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:44:23 2022] DELETE /identity/v3/projects/9fdfbb13c5a34b72b1bb3646a731a3c8 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:23.974961 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e69891f-21ae-471a-926a-738f0d77e5a4 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:23.975850 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e69891f-21ae-471a-926a-738f0d77e5a4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:23.976194 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e69891f-21ae-471a-926a-738f0d77e5a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:23.997809 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-2e69891f-21ae-471a-926a-738f0d77e5a4 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:23.999208 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2e69891f-21ae-471a-926a-738f0d77e5a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:24.072122 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1132/2248] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:44:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:24.080154 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f62813b-0713-4bc0-921d-f98ccc637a8d None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:24.080453 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f62813b-0713-4bc0-921d-f98ccc637a8d None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:24.080698 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f62813b-0713-4bc0-921d-f98ccc637a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:24.105796 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-3f62813b-0713-4bc0-921d-f98ccc637a8d None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:24.107840 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3f62813b-0713-4bc0-921d-f98ccc637a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:24.117283 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1117/2249] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:44:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:24.122234 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e69891f-21ae-471a-926a-738f0d77e5a4 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:24.134097 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e69891f-21ae-471a-926a-738f0d77e5a4 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:24.139599 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a959266-ce01-4e70-8ce8-0136bfb334b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:24.140496 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a959266-ce01-4e70-8ce8-0136bfb334b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:24.140936 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a959266-ce01-4e70-8ce8-0136bfb334b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:24.141412 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a959266-ce01-4e70-8ce8-0136bfb334b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:24.142136 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a959266-ce01-4e70-8ce8-0136bfb334b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:44:24.142446 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a959266-ce01-4e70-8ce8-0136bfb334b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:24.144277 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a959266-ce01-4e70-8ce8-0136bfb334b5 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:24.158564 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1133/2250] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:44:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:44:24.162466 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f62813b-0713-4bc0-921d-f98ccc637a8d None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:24.174465 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f62813b-0713-4bc0-921d-f98ccc637a8d None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:24.180597 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f2e39d-7b4d-4c8f-bf1f-bc5cbfca95ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:24.181323 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2e39d-7b4d-4c8f-bf1f-bc5cbfca95ca admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:24.181698 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2e39d-7b4d-4c8f-bf1f-bc5cbfca95ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:24.182122 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f2e39d-7b4d-4c8f-bf1f-bc5cbfca95ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:24.182861 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f2e39d-7b4d-4c8f-bf1f-bc5cbfca95ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:24.184850 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f2e39d-7b4d-4c8f-bf1f-bc5cbfca95ca admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:24.220871 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1118/2251] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:44:24 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:44:24.225704 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a959266-ce01-4e70-8ce8-0136bfb334b5 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:24.235803 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a959266-ce01-4e70-8ce8-0136bfb334b5 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:24.316100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f690e16e-baae-4c2e-a8a6-bfb6f09cfbf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:24.316709 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690e16e-baae-4c2e-a8a6-bfb6f09cfbf4 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:24.316988 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690e16e-baae-4c2e-a8a6-bfb6f09cfbf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:24.317258 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690e16e-baae-4c2e-a8a6-bfb6f09cfbf4 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:24.317937 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f690e16e-baae-4c2e-a8a6-bfb6f09cfbf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:24.319895 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f690e16e-baae-4c2e-a8a6-bfb6f09cfbf4 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:24.339525 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:24.351026 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1134/2252] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:44:24 2022] POST /identity/v3/users => generated 287 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:44:24.352106 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f2e39d-7b4d-4c8f-bf1f-bc5cbfca95ca admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:24.362629 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f2e39d-7b4d-4c8f-bf1f-bc5cbfca95ca admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:24.367383 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a4e9ec-b490-4c25-b1c9-2e7e04170910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:24.367991 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a4e9ec-b490-4c25-b1c9-2e7e04170910 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:24.368260 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a4e9ec-b490-4c25-b1c9-2e7e04170910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:24.368529 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a4e9ec-b490-4c25-b1c9-2e7e04170910 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:24.369171 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a4e9ec-b490-4c25-b1c9-2e7e04170910 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:24.370794 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a4e9ec-b490-4c25-b1c9-2e7e04170910 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:24.382597 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1119/2253] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:44:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:44:24.386639 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f690e16e-baae-4c2e-a8a6-bfb6f09cfbf4 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:24.398012 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f690e16e-baae-4c2e-a8a6-bfb6f09cfbf4 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:24.404108 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b25aef-94ae-4478-a9ed-bb5197c1f605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:24.404783 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b25aef-94ae-4478-a9ed-bb5197c1f605 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:24.405131 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b25aef-94ae-4478-a9ed-bb5197c1f605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:24.406649 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b25aef-94ae-4478-a9ed-bb5197c1f605 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:24.407302 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b25aef-94ae-4478-a9ed-bb5197c1f605 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:24.408902 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b25aef-94ae-4478-a9ed-bb5197c1f605 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:24.421239 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1135/2254] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:44:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:44:24.425549 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a4e9ec-b490-4c25-b1c9-2e7e04170910 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:24.435741 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a4e9ec-b490-4c25-b1c9-2e7e04170910 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:24.440511 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d32a2a-a407-4412-a537-8a3a37219544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:24.441186 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d32a2a-a407-4412-a537-8a3a37219544 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:24.441462 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d32a2a-a407-4412-a537-8a3a37219544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:24.441733 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d32a2a-a407-4412-a537-8a3a37219544 admin admin] PATH_INFO: `/v3/projects/594469a2b62741a285445e2b7dd6d8ea/users/c3a997daec2748f6bfa0fe3fbef77a29/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:24.456446 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d32a2a-a407-4412-a537-8a3a37219544 admin admin] RBAC: Authorizing `identity:create_grant(project_id=594469a2b62741a285445e2b7dd6d8ea, user_id=c3a997daec2748f6bfa0fe3fbef77a29, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:24.459683 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d32a2a-a407-4412-a537-8a3a37219544 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:24.472968 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1120/2255] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:44:24 2022] PUT /identity/v3/projects/594469a2b62741a285445e2b7dd6d8ea/users/c3a997daec2748f6bfa0fe3fbef77a29/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:44:24.484620 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9a863-af0c-4ef8-a9c0-a03b72afa86e None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:24.484988 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9a863-af0c-4ef8-a9c0-a03b72afa86e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:24.485269 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9a863-af0c-4ef8-a9c0-a03b72afa86e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:24.514650 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-2cc9a863-af0c-4ef8-a9c0-a03b72afa86e None None] MFA Rules not processed for user `c3a997daec2748f6bfa0fe3fbef77a29`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:24.515892 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2cc9a863-af0c-4ef8-a9c0-a03b72afa86e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:24.582232 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1136/2256] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:44:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:44:24.598194 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d32a2a-a407-4412-a537-8a3a37219544 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:24.600431 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-20d32a2a-a407-4412-a537-8a3a37219544 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:24.605758 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:24.666955 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d32a2a-a407-4412-a537-8a3a37219544 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:24.671086 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6c832b-ecd6-40b4-a707-f0650510fe50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:24.671692 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c832b-ecd6-40b4-a707-f0650510fe50 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:24.671967 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c832b-ecd6-40b4-a707-f0650510fe50 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:24.672234 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c832b-ecd6-40b4-a707-f0650510fe50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:24.680911 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6c832b-ecd6-40b4-a707-f0650510fe50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:24.682549 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6c832b-ecd6-40b4-a707-f0650510fe50 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:24.691551 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1121/2257] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:44:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:44:28.496175 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5912888-5a98-4d69-abdd-447b793050f8 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:28.496959 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5912888-5a98-4d69-abdd-447b793050f8 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:28.498225 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5912888-5a98-4d69-abdd-447b793050f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:28.518761 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-a5912888-5a98-4d69-abdd-447b793050f8 None None] MFA Rules not processed for user `c81e5fb007614cf2992f5e02e85d6307`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:28.519989 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a5912888-5a98-4d69-abdd-447b793050f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:28.583758 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1137/2258] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Sep 28 15:44:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:28.596969 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6c832b-ecd6-40b4-a707-f0650510fe50 service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:28.598367 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-8a6c832b-ecd6-40b4-a707-f0650510fe50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:28.669638 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6c832b-ecd6-40b4-a707-f0650510fe50 service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:28.675114 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92080671-97cc-4aaa-9973-e1b269468523 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '328051fe7607464f8f5dc0dcc01a817a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:28.676100 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92080671-97cc-4aaa-9973-e1b269468523 service swift] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:28.676418 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92080671-97cc-4aaa-9973-e1b269468523 service swift] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:28.676721 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92080671-97cc-4aaa-9973-e1b269468523 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:28.686462 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92080671-97cc-4aaa-9973-e1b269468523 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:28.688197 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92080671-97cc-4aaa-9973-e1b269468523 service swift] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:28.697434 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1122/2259] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 15:44:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 15:44:29.995565 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5912888-5a98-4d69-abdd-447b793050f8 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:30.062085 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5912888-5a98-4d69-abdd-447b793050f8 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:30.067387 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05baedc4-e1b8-4642-b1a0-c39f9c50c731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:30.068051 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05baedc4-e1b8-4642-b1a0-c39f9c50c731 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:30.068330 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05baedc4-e1b8-4642-b1a0-c39f9c50c731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:30.068624 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05baedc4-e1b8-4642-b1a0-c39f9c50c731 admin admin] PATH_INFO: `/v3/users/c3a997daec2748f6bfa0fe3fbef77a29` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:30.069773 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05baedc4-e1b8-4642-b1a0-c39f9c50c731 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3a997daec2748f6bfa0fe3fbef77a29)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:30.071866 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05baedc4-e1b8-4642-b1a0-c39f9c50c731 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:30.098756 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:30.129963 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-05baedc4-e1b8-4642-b1a0-c39f9c50c731 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3a997daec2748f6bfa0fe3fbef77a29'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:30.130821 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-05baedc4-e1b8-4642-b1a0-c39f9c50c731 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3a997daec2748f6bfa0fe3fbef77a29'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:30.147296 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-05baedc4-e1b8-4642-b1a0-c39f9c50c731 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3a997daec2748f6bfa0fe3fbef77a29'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:30.156496 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-05baedc4-e1b8-4642-b1a0-c39f9c50c731 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3a997daec2748f6bfa0fe3fbef77a29'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:30.169011 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-05baedc4-e1b8-4642-b1a0-c39f9c50c731 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3a997daec2748f6bfa0fe3fbef77a29'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:30.179707 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1138/2260] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:44:29 2022] DELETE /identity/v3/users/c3a997daec2748f6bfa0fe3fbef77a29 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:30.181425 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d2548a-f73d-49c4-95f0-8f994b2bcc66 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:30.181932 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d2548a-f73d-49c4-95f0-8f994b2bcc66 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:30.182031 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d2548a-f73d-49c4-95f0-8f994b2bcc66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:30.229359 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-e6d2548a-f73d-49c4-95f0-8f994b2bcc66 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:30.230903 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e6d2548a-f73d-49c4-95f0-8f994b2bcc66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:30.336141 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1123/2261] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:44:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:30.345694 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05baedc4-e1b8-4642-b1a0-c39f9c50c731 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:30.347182 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-05baedc4-e1b8-4642-b1a0-c39f9c50c731 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:30.413141 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05baedc4-e1b8-4642-b1a0-c39f9c50c731 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:30.416782 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0191cd4-90da-4907-b737-98ce36e8c600 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:30.417388 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0191cd4-90da-4907-b737-98ce36e8c600 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:30.417685 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0191cd4-90da-4907-b737-98ce36e8c600 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:30.417987 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0191cd4-90da-4907-b737-98ce36e8c600 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:30.427212 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0191cd4-90da-4907-b737-98ce36e8c600 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:30.429129 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0191cd4-90da-4907-b737-98ce36e8c600 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:30.438361 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1139/2262] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:44:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:44:30.728051 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d2548a-f73d-49c4-95f0-8f994b2bcc66 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:30.729557 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e6d2548a-f73d-49c4-95f0-8f994b2bcc66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:30.745182 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d2548a-f73d-49c4-95f0-8f994b2bcc66 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:30.750112 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cf562a-5939-4359-917d-87d986d42417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:30.750746 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf562a-5939-4359-917d-87d986d42417 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:30.751021 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf562a-5939-4359-917d-87d986d42417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:30.751293 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf562a-5939-4359-917d-87d986d42417 admin admin] PATH_INFO: `/v3/projects/594469a2b62741a285445e2b7dd6d8ea` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:30.752547 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cf562a-5939-4359-917d-87d986d42417 admin admin] RBAC: Authorizing `identity:delete_project(project_id=594469a2b62741a285445e2b7dd6d8ea)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:30.754541 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cf562a-5939-4359-917d-87d986d42417 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:30.800819 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:30.815768 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-10cf562a-5939-4359-917d-87d986d42417 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '594469a2b62741a285445e2b7dd6d8ea', 'request_id': 'req-10cf562a-5939-4359-917d-87d986d42417'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:30.830680 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-10cf562a-5939-4359-917d-87d986d42417 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '594469a2b62741a285445e2b7dd6d8ea', 'request_id': 'req-10cf562a-5939-4359-917d-87d986d42417'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:30.843827 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-10cf562a-5939-4359-917d-87d986d42417 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '594469a2b62741a285445e2b7dd6d8ea', 'request_id': 'req-10cf562a-5939-4359-917d-87d986d42417'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:30.855351 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-10cf562a-5939-4359-917d-87d986d42417 admin admin] The token cache is being invalidate because project 594469a2b62741a285445e2b7dd6d8ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:44:30.855638 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-10cf562a-5939-4359-917d-87d986d42417 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:30.856296 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-10cf562a-5939-4359-917d-87d986d42417 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:30.857765 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1124/2263] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:44:30 2022] DELETE /identity/v3/projects/594469a2b62741a285445e2b7dd6d8ea => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:30.862676 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ab566-9683-4dd1-b44f-8d60805a4f42 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:30.862966 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ab566-9683-4dd1-b44f-8d60805a4f42 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:30.863244 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ab566-9683-4dd1-b44f-8d60805a4f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:30.883100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-f59ab566-9683-4dd1-b44f-8d60805a4f42 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:30.884293 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f59ab566-9683-4dd1-b44f-8d60805a4f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:30.950499 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1140/2264] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:44:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:30.960353 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d01c4-9e89-4f93-91c5-8eece67fc4c9 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:30.960770 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d01c4-9e89-4f93-91c5-8eece67fc4c9 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:30.961062 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d01c4-9e89-4f93-91c5-8eece67fc4c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:30.982275 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-050d01c4-9e89-4f93-91c5-8eece67fc4c9 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:30.983557 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-050d01c4-9e89-4f93-91c5-8eece67fc4c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:30.992177 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1125/2265] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:44:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:30.996929 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ab566-9683-4dd1-b44f-8d60805a4f42 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:31.006956 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ab566-9683-4dd1-b44f-8d60805a4f42 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:31.011993 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab11be1e-10d0-4316-afac-d7794a990480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:31.012589 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab11be1e-10d0-4316-afac-d7794a990480 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:31.012870 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab11be1e-10d0-4316-afac-d7794a990480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:31.013140 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab11be1e-10d0-4316-afac-d7794a990480 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:31.013824 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab11be1e-10d0-4316-afac-d7794a990480 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:44:31.014141 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab11be1e-10d0-4316-afac-d7794a990480 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:31.015740 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab11be1e-10d0-4316-afac-d7794a990480 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:31.032899 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1141/2266] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:44:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:44:31.034677 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d01c4-9e89-4f93-91c5-8eece67fc4c9 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:31.047259 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d01c4-9e89-4f93-91c5-8eece67fc4c9 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:31.056172 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28dfec9-cfad-40a4-8aba-c0ee7f5abb50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:31.056860 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28dfec9-cfad-40a4-8aba-c0ee7f5abb50 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:31.057205 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28dfec9-cfad-40a4-8aba-c0ee7f5abb50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:31.057410 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28dfec9-cfad-40a4-8aba-c0ee7f5abb50 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:31.058194 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28dfec9-cfad-40a4-8aba-c0ee7f5abb50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:31.060787 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28dfec9-cfad-40a4-8aba-c0ee7f5abb50 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:31.093303 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1126/2267] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:44:31 2022] POST /identity/v3/projects => generated 391 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:44:31.097516 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab11be1e-10d0-4316-afac-d7794a990480 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:31.127454 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab11be1e-10d0-4316-afac-d7794a990480 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:31.156420 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:31.187113 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b420cf-c242-4865-8982-33e39bd36fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:31.187725 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b420cf-c242-4865-8982-33e39bd36fdf admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:31.188002 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b420cf-c242-4865-8982-33e39bd36fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:31.188449 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b420cf-c242-4865-8982-33e39bd36fdf admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:31.189144 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b420cf-c242-4865-8982-33e39bd36fdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:31.191216 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b420cf-c242-4865-8982-33e39bd36fdf admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:31.216217 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1142/2268] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:44:31 2022] POST /identity/v3/users => generated 296 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:44:31.220345 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28dfec9-cfad-40a4-8aba-c0ee7f5abb50 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:31.230608 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28dfec9-cfad-40a4-8aba-c0ee7f5abb50 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:31.235066 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d669f26-e64f-4731-8f29-0f880ef3ca22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:31.235653 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d669f26-e64f-4731-8f29-0f880ef3ca22 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:31.235926 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d669f26-e64f-4731-8f29-0f880ef3ca22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:31.236192 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d669f26-e64f-4731-8f29-0f880ef3ca22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:31.236822 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d669f26-e64f-4731-8f29-0f880ef3ca22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:31.238436 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d669f26-e64f-4731-8f29-0f880ef3ca22 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:31.251427 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1127/2269] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:44:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:44:31.255277 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b420cf-c242-4865-8982-33e39bd36fdf admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:31.266000 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b420cf-c242-4865-8982-33e39bd36fdf admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:31.276317 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60945dc-e3da-4f4d-a1fb-8cef9339cded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:31.276948 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60945dc-e3da-4f4d-a1fb-8cef9339cded admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:31.277221 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60945dc-e3da-4f4d-a1fb-8cef9339cded admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:31.277488 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60945dc-e3da-4f4d-a1fb-8cef9339cded admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:31.278162 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60945dc-e3da-4f4d-a1fb-8cef9339cded admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:31.279811 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60945dc-e3da-4f4d-a1fb-8cef9339cded admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:31.291391 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1143/2270] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:44:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:44:31.295518 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d669f26-e64f-4731-8f29-0f880ef3ca22 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:31.305954 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d669f26-e64f-4731-8f29-0f880ef3ca22 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:31.310476 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c163267e-6128-4693-902e-310b00ea8aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:31.311114 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c163267e-6128-4693-902e-310b00ea8aae admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:31.311387 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c163267e-6128-4693-902e-310b00ea8aae admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:31.311654 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c163267e-6128-4693-902e-310b00ea8aae admin admin] PATH_INFO: `/v3/projects/69e3cb0e89df473ba66a12f3d07b948d/users/9a5860a782ad4a769cd7edb0dd40236e/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:31.328367 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c163267e-6128-4693-902e-310b00ea8aae admin admin] RBAC: Authorizing `identity:create_grant(project_id=69e3cb0e89df473ba66a12f3d07b948d, user_id=9a5860a782ad4a769cd7edb0dd40236e, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:31.331064 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c163267e-6128-4693-902e-310b00ea8aae admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:31.357283 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1128/2271] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:44:31 2022] PUT /identity/v3/projects/69e3cb0e89df473ba66a12f3d07b948d/users/9a5860a782ad4a769cd7edb0dd40236e/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:31.358574 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c55e9d-13eb-4e47-a391-fadf4b4547ef None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:31.358925 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c55e9d-13eb-4e47-a391-fadf4b4547ef None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:31.359225 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c55e9d-13eb-4e47-a391-fadf4b4547ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:31.390429 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-94c55e9d-13eb-4e47-a391-fadf4b4547ef None None] MFA Rules not processed for user `9a5860a782ad4a769cd7edb0dd40236e`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:31.391657 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-94c55e9d-13eb-4e47-a391-fadf4b4547ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:31.489912 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1144/2272] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:44:31 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:44:31.501421 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c163267e-6128-4693-902e-310b00ea8aae admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:31.503398 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c163267e-6128-4693-902e-310b00ea8aae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:31.570437 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c163267e-6128-4693-902e-310b00ea8aae admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:31.574230 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb5f723-8db7-4861-a03f-ff25b411348a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:31.574840 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5f723-8db7-4861-a03f-ff25b411348a service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:31.575120 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5f723-8db7-4861-a03f-ff25b411348a service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:31.575393 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5f723-8db7-4861-a03f-ff25b411348a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:31.584422 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb5f723-8db7-4861-a03f-ff25b411348a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:31.585878 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb5f723-8db7-4861-a03f-ff25b411348a service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:31.595914 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1129/2273] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:44:31 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:44:41.485472 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=52,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:41.591896 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=66,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:46.375456 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c55e9d-13eb-4e47-a391-fadf4b4547ef None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:46.376858 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-94c55e9d-13eb-4e47-a391-fadf4b4547ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:46.443592 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c55e9d-13eb-4e47-a391-fadf4b4547ef None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:46.448530 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e68bba-33a2-4699-a134-9cc285ca0d12 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de934d0105bd4d01ba6c06a4a0d36704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:46.449133 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e68bba-33a2-4699-a134-9cc285ca0d12 service placement] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:46.449409 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e68bba-33a2-4699-a134-9cc285ca0d12 service placement] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:46.449708 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e68bba-33a2-4699-a134-9cc285ca0d12 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:46.451433 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d8e68bba-33a2-4699-a134-9cc285ca0d12 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:46.463012 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e68bba-33a2-4699-a134-9cc285ca0d12 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:46.464571 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e68bba-33a2-4699-a134-9cc285ca0d12 service placement] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:46.474120 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1145/2274] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:44:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:44:52.186733 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb5f723-8db7-4861-a03f-ff25b411348a service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:52.250885 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb5f723-8db7-4861-a03f-ff25b411348a service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:52.255404 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa2ebf6-c518-4395-b481-80ec78259b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:52.256034 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2ebf6-c518-4395-b481-80ec78259b30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:52.256309 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2ebf6-c518-4395-b481-80ec78259b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:52.256577 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2ebf6-c518-4395-b481-80ec78259b30 admin admin] PATH_INFO: `/v3/users/9a5860a782ad4a769cd7edb0dd40236e` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:52.257693 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa2ebf6-c518-4395-b481-80ec78259b30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a5860a782ad4a769cd7edb0dd40236e)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:52.259644 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa2ebf6-c518-4395-b481-80ec78259b30 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:52.308582 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-cfa2ebf6-c518-4395-b481-80ec78259b30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a5860a782ad4a769cd7edb0dd40236e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:52.315296 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-cfa2ebf6-c518-4395-b481-80ec78259b30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a5860a782ad4a769cd7edb0dd40236e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:52.315950 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-cfa2ebf6-c518-4395-b481-80ec78259b30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a5860a782ad4a769cd7edb0dd40236e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:52.323378 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-cfa2ebf6-c518-4395-b481-80ec78259b30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a5860a782ad4a769cd7edb0dd40236e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:52.323998 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-cfa2ebf6-c518-4395-b481-80ec78259b30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a5860a782ad4a769cd7edb0dd40236e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:52.341129 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1130/2275] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:44:52 2022] DELETE /identity/v3/users/9a5860a782ad4a769cd7edb0dd40236e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:52.345699 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2af5f-20c0-4a34-85f9-a1c0013d20d6 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:52.346213 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2af5f-20c0-4a34-85f9-a1c0013d20d6 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:52.346305 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2af5f-20c0-4a34-85f9-a1c0013d20d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:52.365470 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-17e2af5f-20c0-4a34-85f9-a1c0013d20d6 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:52.366761 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-17e2af5f-20c0-4a34-85f9-a1c0013d20d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:52.428153 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1146/2276] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:44:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:52.437759 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa2ebf6-c518-4395-b481-80ec78259b30 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:52.439191 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-cfa2ebf6-c518-4395-b481-80ec78259b30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:52.495626 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa2ebf6-c518-4395-b481-80ec78259b30 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:52.499543 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27108f2-4a8b-4185-8533-67b93e530232 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:52.500141 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27108f2-4a8b-4185-8533-67b93e530232 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:52.500415 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27108f2-4a8b-4185-8533-67b93e530232 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:52.500680 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27108f2-4a8b-4185-8533-67b93e530232 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:52.509538 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27108f2-4a8b-4185-8533-67b93e530232 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:52.511824 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27108f2-4a8b-4185-8533-67b93e530232 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:52.526087 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1131/2277] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:44:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:44:52.787268 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2af5f-20c0-4a34-85f9-a1c0013d20d6 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:52.788795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-17e2af5f-20c0-4a34-85f9-a1c0013d20d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:52.800472 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2af5f-20c0-4a34-85f9-a1c0013d20d6 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:52.806027 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21357a7-0bc0-4164-b506-29bd56d1c17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:52.806674 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21357a7-0bc0-4164-b506-29bd56d1c17e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:52.806946 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21357a7-0bc0-4164-b506-29bd56d1c17e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:52.807244 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21357a7-0bc0-4164-b506-29bd56d1c17e admin admin] PATH_INFO: `/v3/projects/69e3cb0e89df473ba66a12f3d07b948d` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:52.808381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21357a7-0bc0-4164-b506-29bd56d1c17e admin admin] RBAC: Authorizing `identity:delete_project(project_id=69e3cb0e89df473ba66a12f3d07b948d)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:52.810350 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21357a7-0bc0-4164-b506-29bd56d1c17e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:52.862791 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c21357a7-0bc0-4164-b506-29bd56d1c17e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69e3cb0e89df473ba66a12f3d07b948d', 'request_id': 'req-c21357a7-0bc0-4164-b506-29bd56d1c17e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:52.869052 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c21357a7-0bc0-4164-b506-29bd56d1c17e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69e3cb0e89df473ba66a12f3d07b948d', 'request_id': 'req-c21357a7-0bc0-4164-b506-29bd56d1c17e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:52.876797 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c21357a7-0bc0-4164-b506-29bd56d1c17e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69e3cb0e89df473ba66a12f3d07b948d', 'request_id': 'req-c21357a7-0bc0-4164-b506-29bd56d1c17e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:52.888847 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c21357a7-0bc0-4164-b506-29bd56d1c17e admin admin] The token cache is being invalidate because project 69e3cb0e89df473ba66a12f3d07b948d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:44:52.889133 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c21357a7-0bc0-4164-b506-29bd56d1c17e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:52.889765 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c21357a7-0bc0-4164-b506-29bd56d1c17e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:52.891258 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1147/2278] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:44:52 2022] DELETE /identity/v3/projects/69e3cb0e89df473ba66a12f3d07b948d => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:56.420173 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27108f2-4a8b-4185-8533-67b93e530232 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:56.421538 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b27108f2-4a8b-4185-8533-67b93e530232 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:56.434592 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27108f2-4a8b-4185-8533-67b93e530232 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:56.452288 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:56.497422 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8726f18-ad69-4036-83dc-9ec2f061c563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:56.498083 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8726f18-ad69-4036-83dc-9ec2f061c563 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:56.498358 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8726f18-ad69-4036-83dc-9ec2f061c563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:56.498623 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8726f18-ad69-4036-83dc-9ec2f061c563 admin admin] PATH_INFO: `/v3/users/bd7995d56920468c8da40fb74e5e14e7` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:56.499719 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8726f18-ad69-4036-83dc-9ec2f061c563 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd7995d56920468c8da40fb74e5e14e7)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:56.501611 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8726f18-ad69-4036-83dc-9ec2f061c563 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:56.555325 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e8726f18-ad69-4036-83dc-9ec2f061c563 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd7995d56920468c8da40fb74e5e14e7'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:56.562835 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e8726f18-ad69-4036-83dc-9ec2f061c563 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd7995d56920468c8da40fb74e5e14e7'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:56.563546 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e8726f18-ad69-4036-83dc-9ec2f061c563 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd7995d56920468c8da40fb74e5e14e7'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:56.575641 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e8726f18-ad69-4036-83dc-9ec2f061c563 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd7995d56920468c8da40fb74e5e14e7'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:56.576403 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e8726f18-ad69-4036-83dc-9ec2f061c563 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd7995d56920468c8da40fb74e5e14e7'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:56.595535 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1132/2279] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:44:56 2022] DELETE /identity/v3/users/bd7995d56920468c8da40fb74e5e14e7 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:56.600763 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f2f61-2260-4b39-9948-82d126712d80 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:56.600763 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f2f61-2260-4b39-9948-82d126712d80 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:56.601435 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f2f61-2260-4b39-9948-82d126712d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:56.621253 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-a95f2f61-2260-4b39-9948-82d126712d80 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:56.621930 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a95f2f61-2260-4b39-9948-82d126712d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:56.630688 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:56.691310 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1148/2280] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:44:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:56.697510 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8726f18-ad69-4036-83dc-9ec2f061c563 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:56.698963 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e8726f18-ad69-4036-83dc-9ec2f061c563 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:56.756706 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8726f18-ad69-4036-83dc-9ec2f061c563 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:56.760407 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da3cf8-5f73-4455-90d1-7407a755e37e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:56.761008 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da3cf8-5f73-4455-90d1-7407a755e37e service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:56.761285 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da3cf8-5f73-4455-90d1-7407a755e37e service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:56.761555 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da3cf8-5f73-4455-90d1-7407a755e37e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:56.770723 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85da3cf8-5f73-4455-90d1-7407a755e37e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:56.772659 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85da3cf8-5f73-4455-90d1-7407a755e37e service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:56.782016 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1133/2281] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:44:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:44:57.039454 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95f2f61-2260-4b39-9948-82d126712d80 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:57.040870 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a95f2f61-2260-4b39-9948-82d126712d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:57.052695 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95f2f61-2260-4b39-9948-82d126712d80 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:57.058016 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32be4c3-ff63-416e-8aaf-15ee88f0c2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:57.058649 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32be4c3-ff63-416e-8aaf-15ee88f0c2d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:57.058926 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32be4c3-ff63-416e-8aaf-15ee88f0c2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:57.059195 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32be4c3-ff63-416e-8aaf-15ee88f0c2d9 admin admin] PATH_INFO: `/v3/projects/e4267acfa5ad411c9b132c49946b67fd` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:57.060497 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32be4c3-ff63-416e-8aaf-15ee88f0c2d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4267acfa5ad411c9b132c49946b67fd)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:57.062460 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32be4c3-ff63-416e-8aaf-15ee88f0c2d9 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:57.109315 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b32be4c3-ff63-416e-8aaf-15ee88f0c2d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4267acfa5ad411c9b132c49946b67fd', 'request_id': 'req-b32be4c3-ff63-416e-8aaf-15ee88f0c2d9'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:57.116361 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b32be4c3-ff63-416e-8aaf-15ee88f0c2d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4267acfa5ad411c9b132c49946b67fd', 'request_id': 'req-b32be4c3-ff63-416e-8aaf-15ee88f0c2d9'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:57.124449 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b32be4c3-ff63-416e-8aaf-15ee88f0c2d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4267acfa5ad411c9b132c49946b67fd', 'request_id': 'req-b32be4c3-ff63-416e-8aaf-15ee88f0c2d9'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:57.135204 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b32be4c3-ff63-416e-8aaf-15ee88f0c2d9 admin admin] The token cache is being invalidate because project e4267acfa5ad411c9b132c49946b67fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:44:57.135489 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b32be4c3-ff63-416e-8aaf-15ee88f0c2d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:57.136112 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b32be4c3-ff63-416e-8aaf-15ee88f0c2d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:44:57.137581 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1149/2282] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:44:57 2022] DELETE /identity/v3/projects/e4267acfa5ad411c9b132c49946b67fd => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:57.142550 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc2484-7e0f-4987-9d6f-0f692db84f61 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:57.142832 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc2484-7e0f-4987-9d6f-0f692db84f61 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:57.143108 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc2484-7e0f-4987-9d6f-0f692db84f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:57.163049 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-04dc2484-7e0f-4987-9d6f-0f692db84f61 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:57.164237 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-04dc2484-7e0f-4987-9d6f-0f692db84f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:57.204679 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:57.225407 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1134/2283] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:44:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:57.233081 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831ceb26-b2dd-4270-bac5-76fbbad7691c None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:57.233370 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831ceb26-b2dd-4270-bac5-76fbbad7691c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:57.233652 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831ceb26-b2dd-4270-bac5-76fbbad7691c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:57.255175 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-831ceb26-b2dd-4270-bac5-76fbbad7691c None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:57.256981 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-831ceb26-b2dd-4270-bac5-76fbbad7691c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:57.264959 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1150/2284] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:44:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:57.269205 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dc2484-7e0f-4987-9d6f-0f692db84f61 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:57.279906 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dc2484-7e0f-4987-9d6f-0f692db84f61 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:57.284571 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c39dee-ba6f-4c32-bfab-b873fc968f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:57.285164 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c39dee-ba6f-4c32-bfab-b873fc968f7f admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:57.285437 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c39dee-ba6f-4c32-bfab-b873fc968f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:57.285701 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c39dee-ba6f-4c32-bfab-b873fc968f7f admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:57.286398 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c39dee-ba6f-4c32-bfab-b873fc968f7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:44:57.286691 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c39dee-ba6f-4c32-bfab-b873fc968f7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:57.288287 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c39dee-ba6f-4c32-bfab-b873fc968f7f admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:57.300033 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1135/2285] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:44:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:44:57.304441 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831ceb26-b2dd-4270-bac5-76fbbad7691c None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:57.313623 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831ceb26-b2dd-4270-bac5-76fbbad7691c None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:57.318218 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb57ed3-a2a9-49ab-91ba-21f3a487b104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:57.319452 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb57ed3-a2a9-49ab-91ba-21f3a487b104 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:57.319730 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb57ed3-a2a9-49ab-91ba-21f3a487b104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:57.319998 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb57ed3-a2a9-49ab-91ba-21f3a487b104 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:57.320624 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb57ed3-a2a9-49ab-91ba-21f3a487b104 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:57.322541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb57ed3-a2a9-49ab-91ba-21f3a487b104 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:57.348627 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1151/2286] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:44:57 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:44:57.354051 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c39dee-ba6f-4c32-bfab-b873fc968f7f admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:57.365072 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c39dee-ba6f-4c32-bfab-b873fc968f7f admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:57.421698 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc1a1a-de5a-46af-a915-4cf3ce002ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:57.422341 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc1a1a-de5a-46af-a915-4cf3ce002ae0 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:57.422530 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc1a1a-de5a-46af-a915-4cf3ce002ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:57.422809 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc1a1a-de5a-46af-a915-4cf3ce002ae0 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:57.423443 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcc1a1a-de5a-46af-a915-4cf3ce002ae0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:57.425350 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcc1a1a-de5a-46af-a915-4cf3ce002ae0 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:57.451704 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1136/2287] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:44:57 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:44:57.453163 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb57ed3-a2a9-49ab-91ba-21f3a487b104 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:57.463716 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb57ed3-a2a9-49ab-91ba-21f3a487b104 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:57.468196 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a7497-d761-463e-95d1-d1879b93dc8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:57.468775 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a7497-d761-463e-95d1-d1879b93dc8f admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:57.469050 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a7497-d761-463e-95d1-d1879b93dc8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:57.469318 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a7497-d761-463e-95d1-d1879b93dc8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:57.470029 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55a7497-d761-463e-95d1-d1879b93dc8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:57.471633 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55a7497-d761-463e-95d1-d1879b93dc8f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:57.482186 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1152/2288] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:44:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:44:57.486105 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc1a1a-de5a-46af-a915-4cf3ce002ae0 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:57.497133 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcc1a1a-de5a-46af-a915-4cf3ce002ae0 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:57.501725 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272cba20-35ef-4fff-bba9-aeefd3ec650a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:57.502355 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272cba20-35ef-4fff-bba9-aeefd3ec650a admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:57.502631 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272cba20-35ef-4fff-bba9-aeefd3ec650a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:57.502921 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272cba20-35ef-4fff-bba9-aeefd3ec650a admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:57.503562 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272cba20-35ef-4fff-bba9-aeefd3ec650a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:57.505162 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272cba20-35ef-4fff-bba9-aeefd3ec650a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:57.515731 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1137/2289] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:44:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:44:57.519798 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a7497-d761-463e-95d1-d1879b93dc8f admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:57.530054 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a7497-d761-463e-95d1-d1879b93dc8f admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:57.534581 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d193e617-55e6-4d51-9ff5-4162c75dfec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:57.535215 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193e617-55e6-4d51-9ff5-4162c75dfec6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:57.535492 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193e617-55e6-4d51-9ff5-4162c75dfec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:57.535761 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193e617-55e6-4d51-9ff5-4162c75dfec6 admin admin] PATH_INFO: `/v3/projects/6dcadaaec3664e8dace58c2479d8f534/users/de27b2da998a4728b07fccfb12294e3d/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:57.550310 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d193e617-55e6-4d51-9ff5-4162c75dfec6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dcadaaec3664e8dace58c2479d8f534, user_id=de27b2da998a4728b07fccfb12294e3d, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:57.552918 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d193e617-55e6-4d51-9ff5-4162c75dfec6 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:57.565653 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1153/2290] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:44:57 2022] PUT /identity/v3/projects/6dcadaaec3664e8dace58c2479d8f534/users/de27b2da998a4728b07fccfb12294e3d/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:44:57.570657 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2566f5f-d0a2-4111-8048-b6d0aa5f6760 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:57.570940 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2566f5f-d0a2-4111-8048-b6d0aa5f6760 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:57.571212 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2566f5f-d0a2-4111-8048-b6d0aa5f6760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:57.590124 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-b2566f5f-d0a2-4111-8048-b6d0aa5f6760 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:57.591317 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b2566f5f-d0a2-4111-8048-b6d0aa5f6760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:57.648734 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1138/2291] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:44:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:44:57.658375 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d193e617-55e6-4d51-9ff5-4162c75dfec6 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:57.660046 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d193e617-55e6-4d51-9ff5-4162c75dfec6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:57.694222 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:57.717607 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d193e617-55e6-4d51-9ff5-4162c75dfec6 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:57.721424 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be1089-5b87-4ef2-99be-e0d68824dfde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:57.722062 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be1089-5b87-4ef2-99be-e0d68824dfde service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:57.722346 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be1089-5b87-4ef2-99be-e0d68824dfde service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:57.722616 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be1089-5b87-4ef2-99be-e0d68824dfde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:57.732320 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34be1089-5b87-4ef2-99be-e0d68824dfde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:57.733758 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34be1089-5b87-4ef2-99be-e0d68824dfde service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:57.743130 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1154/2292] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:44:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:44:58.579614 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a52c310-6d57-413c-b80d-08064765a76b None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:58.580139 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a52c310-6d57-413c-b80d-08064765a76b None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:58.580279 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a52c310-6d57-413c-b80d-08064765a76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:58.610660 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-1a52c310-6d57-413c-b80d-08064765a76b None None] MFA Rules not processed for user `de27b2da998a4728b07fccfb12294e3d`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:44:58.611897 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-1a52c310-6d57-413c-b80d-08064765a76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:58.654061 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:44:58.679261 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1139/2293] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:44:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:44:58.690133 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be1089-5b87-4ef2-99be-e0d68824dfde service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:44:58.691579 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-34be1089-5b87-4ef2-99be-e0d68824dfde service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:44:58.753492 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be1089-5b87-4ef2-99be-e0d68824dfde service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:44:58.759732 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaf51f0-7aae-4b73-9d39-4e289279e1f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:44:58.761274 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf51f0-7aae-4b73-9d39-4e289279e1f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:44:58.761776 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf51f0-7aae-4b73-9d39-4e289279e1f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:44:58.762287 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf51f0-7aae-4b73-9d39-4e289279e1f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:44:58.773617 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaf51f0-7aae-4b73-9d39-4e289279e1f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:44:58.775620 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaf51f0-7aae-4b73-9d39-4e289279e1f0 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:44:58.794178 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1155/2294] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:44:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:45:00.182499 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b74a02c-77ce-4004-bbc9-95a9d045a475 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:00.183093 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b74a02c-77ce-4004-bbc9-95a9d045a475 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:00.183093 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b74a02c-77ce-4004-bbc9-95a9d045a475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:00.201776 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-4b74a02c-77ce-4004-bbc9-95a9d045a475 None None] MFA Rules not processed for user `de27b2da998a4728b07fccfb12294e3d`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:45:00.203007 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4b74a02c-77ce-4004-bbc9-95a9d045a475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:45:00.209809 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1140/2295] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:45:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:45:00.394625 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279c9de-9574-4899-b1d0-9c527a97d99b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:00.395124 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279c9de-9574-4899-b1d0-9c527a97d99b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:00.395308 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279c9de-9574-4899-b1d0-9c527a97d99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:00.415569 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-3279c9de-9574-4899-b1d0-9c527a97d99b None None] MFA Rules not processed for user `c81e5fb007614cf2992f5e02e85d6307`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:45:00.416758 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3279c9de-9574-4899-b1d0-9c527a97d99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:45:00.471445 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1156/2296] 10.222.0.52 () {58 vars in 967 bytes} [Wed Sep 28 15:45:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:45:00.484388 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b74a02c-77ce-4004-bbc9-95a9d045a475 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:45:00.485816 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-4b74a02c-77ce-4004-bbc9-95a9d045a475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:45:00.556179 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b74a02c-77ce-4004-bbc9-95a9d045a475 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:45:00.561114 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb498217-c9a2-4609-8a55-cafed8db718e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '328051fe7607464f8f5dc0dcc01a817a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:45:00.561774 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb498217-c9a2-4609-8a55-cafed8db718e service swift] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:00.562159 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb498217-c9a2-4609-8a55-cafed8db718e service swift] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:00.562342 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb498217-c9a2-4609-8a55-cafed8db718e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:00.570393 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb498217-c9a2-4609-8a55-cafed8db718e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:45:00.571987 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb498217-c9a2-4609-8a55-cafed8db718e service swift] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:45:00.580938 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1141/2297] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 15:45:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 15:45:10.464795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=62 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:45:10.577625 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=44 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:45:23.349858 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8182f28-eec4-4003-9ca0-1b806fbc9c47 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:23.350330 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8182f28-eec4-4003-9ca0-1b806fbc9c47 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:23.350425 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8182f28-eec4-4003-9ca0-1b806fbc9c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:23.371235 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-c8182f28-eec4-4003-9ca0-1b806fbc9c47 None None] MFA Rules not processed for user `f22b26ed7a47480e8448175517170172`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:45:23.372465 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c8182f28-eec4-4003-9ca0-1b806fbc9c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:45:23.432602 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1157/2298] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:45:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:45:33.424663 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=25 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:45:54.138482 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a92103-0c22-4d83-b3b4-095947292b5c None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:45:54.139027 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a92103-0c22-4d83-b3b4-095947292b5c None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:45:54.139027 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a92103-0c22-4d83-b3b4-095947292b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:45:54.157305 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-a6a92103-0c22-4d83-b3b4-095947292b5c None None] MFA Rules not processed for user `36515ae6f91748babdcf6a9a72f59ed1`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:45:54.158578 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a6a92103-0c22-4d83-b3b4-095947292b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:45:54.212054 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1142/2299] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:45:54 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:46:04.205377 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=25 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:46:08.626023 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31917c5-4649-4320-a1b2-d8cd5ea552da None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:08.626555 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31917c5-4649-4320-a1b2-d8cd5ea552da None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:08.626555 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31917c5-4649-4320-a1b2-d8cd5ea552da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:08.645816 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-a31917c5-4649-4320-a1b2-d8cd5ea552da None None] MFA Rules not processed for user `de27b2da998a4728b07fccfb12294e3d`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:46:08.647037 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a31917c5-4649-4320-a1b2-d8cd5ea552da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:08.653988 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1158/2300] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:46:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:46:13.719099 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f301bcd2-ee86-4c2c-b6e8-0aee2462af2f None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:13.719597 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f301bcd2-ee86-4c2c-b6e8-0aee2462af2f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:13.719699 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f301bcd2-ee86-4c2c-b6e8-0aee2462af2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:13.739292 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-f301bcd2-ee86-4c2c-b6e8-0aee2462af2f None None] MFA Rules not processed for user `298dfca23e004969a0c907e18c7237a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:46:13.740787 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-f301bcd2-ee86-4c2c-b6e8-0aee2462af2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:13.796433 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1143/2301] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:46:13 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:46:18.640437 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:46:23.789572 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=25 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:46:27.494243 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31917c5-4649-4320-a1b2-d8cd5ea552da None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:27.495811 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a31917c5-4649-4320-a1b2-d8cd5ea552da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:27.508763 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31917c5-4649-4320-a1b2-d8cd5ea552da None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:27.512305 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f4838a-ba68-4b8d-b258-d3977c14b4e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:27.512908 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f4838a-ba68-4b8d-b258-d3977c14b4e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:27.513190 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f4838a-ba68-4b8d-b258-d3977c14b4e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:27.513466 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f4838a-ba68-4b8d-b258-d3977c14b4e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:27.515243 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-19f4838a-ba68-4b8d-b258-d3977c14b4e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:27.588908 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f4838a-ba68-4b8d-b258-d3977c14b4e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:27.590396 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f4838a-ba68-4b8d-b258-d3977c14b4e4 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:27.601081 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1159/2302] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:46:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:46:28.794339 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f301bcd2-ee86-4c2c-b6e8-0aee2462af2f None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:28.807094 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f301bcd2-ee86-4c2c-b6e8-0aee2462af2f None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:28.813338 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c56ecd5-87e0-41bd-a030-dc54e1830619 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:28.813974 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56ecd5-87e0-41bd-a030-dc54e1830619 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:28.814254 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56ecd5-87e0-41bd-a030-dc54e1830619 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:28.814525 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56ecd5-87e0-41bd-a030-dc54e1830619 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:28.816284 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-2c56ecd5-87e0-41bd-a030-dc54e1830619 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:28.830683 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c56ecd5-87e0-41bd-a030-dc54e1830619 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:28.832107 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c56ecd5-87e0-41bd-a030-dc54e1830619 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:28.845313 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1144/2303] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:46:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:46:36.452987 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f4838a-ba68-4b8d-b258-d3977c14b4e4 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:36.454850 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-19f4838a-ba68-4b8d-b258-d3977c14b4e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:36.456299 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee1520-3ec5-4781-8733-a96f32caa9ee None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:36.456804 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee1520-3ec5-4781-8733-a96f32caa9ee None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:36.457300 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee1520-3ec5-4781-8733-a96f32caa9ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:36.471308 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f4838a-ba68-4b8d-b258-d3977c14b4e4 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:36.475309 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f8ee9b-a328-4836-8499-f9850c407a85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:36.475929 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8ee9b-a328-4836-8499-f9850c407a85 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:36.476198 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8ee9b-a328-4836-8499-f9850c407a85 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:36.476474 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8ee9b-a328-4836-8499-f9850c407a85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:36.478230 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e4f8ee9b-a328-4836-8499-f9850c407a85 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:36.478465 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-94ee1520-3ec5-4781-8733-a96f32caa9ee None None] MFA Rules not processed for user `634c0af05b334b488f2fa358e574172b`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:46:36.479617 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-94ee1520-3ec5-4781-8733-a96f32caa9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:36.489547 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f8ee9b-a328-4836-8499-f9850c407a85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:36.491003 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f8ee9b-a328-4836-8499-f9850c407a85 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:36.500361 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1160/2304] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:46:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:46:36.535205 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1145/2305] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:46:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:46:41.798812 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f8ee9b-a328-4836-8499-f9850c407a85 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:41.800529 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e4f8ee9b-a328-4836-8499-f9850c407a85 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:41.816741 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f8ee9b-a328-4836-8499-f9850c407a85 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:41.833673 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246b83c2-5b89-4f47-bc71-86dd8e49d302 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf14b64bf04b2ba74d2fe1fd41ff71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:41.834732 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b83c2-5b89-4f47-bc71-86dd8e49d302 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:41.835244 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b83c2-5b89-4f47-bc71-86dd8e49d302 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:41.836013 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b83c2-5b89-4f47-bc71-86dd8e49d302 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:41.837817 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-246b83c2-5b89-4f47-bc71-86dd8e49d302 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:41.854936 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246b83c2-5b89-4f47-bc71-86dd8e49d302 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:41.857708 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246b83c2-5b89-4f47-bc71-86dd8e49d302 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:41.884391 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1161/2306] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:46:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:46:46.528401 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=31 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:46:48.848047 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ee1520-3ec5-4781-8733-a96f32caa9ee None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:48.858653 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ee1520-3ec5-4781-8733-a96f32caa9ee None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:48.862199 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91204c77-5a60-492e-b3e8-2578fdfa0552 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:48.862800 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91204c77-5a60-492e-b3e8-2578fdfa0552 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:48.863073 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91204c77-5a60-492e-b3e8-2578fdfa0552 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:48.863340 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91204c77-5a60-492e-b3e8-2578fdfa0552 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:48.865037 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-91204c77-5a60-492e-b3e8-2578fdfa0552 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:48.876731 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91204c77-5a60-492e-b3e8-2578fdfa0552 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:48.878164 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91204c77-5a60-492e-b3e8-2578fdfa0552 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:48.887423 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1146/2307] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:46:48 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:46:51.879104 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=46 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:46:58.882866 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=6 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:46:59.363557 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83543e-d3a2-4ef6-b4f7-8fe244cbc06e None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:59.364087 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83543e-d3a2-4ef6-b4f7-8fe244cbc06e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:59.364174 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83543e-d3a2-4ef6-b4f7-8fe244cbc06e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:59.384169 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-5d83543e-d3a2-4ef6-b4f7-8fe244cbc06e None None] MFA Rules not processed for user `f22b26ed7a47480e8448175517170172`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:46:59.385367 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5d83543e-d3a2-4ef6-b4f7-8fe244cbc06e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:59.392565 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1162/2308] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:46:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:46:59.717231 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91204c77-5a60-492e-b3e8-2578fdfa0552 service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:46:59.726892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91204c77-5a60-492e-b3e8-2578fdfa0552 service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:46:59.731772 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c03aa7-30f2-44f2-b72a-93d508be9b78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:46:59.732890 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c03aa7-30f2-44f2-b72a-93d508be9b78 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:46:59.733293 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c03aa7-30f2-44f2-b72a-93d508be9b78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:46:59.733615 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c03aa7-30f2-44f2-b72a-93d508be9b78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:46:59.736455 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a7c03aa7-30f2-44f2-b72a-93d508be9b78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:46:59.748201 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c03aa7-30f2-44f2-b72a-93d508be9b78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:46:59.749664 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c03aa7-30f2-44f2-b72a-93d508be9b78 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:46:59.757737 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1147/2309] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:46:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:47:00.084978 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d83543e-d3a2-4ef6-b4f7-8fe244cbc06e None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:00.087088 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5d83543e-d3a2-4ef6-b4f7-8fe244cbc06e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:00.182442 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d83543e-d3a2-4ef6-b4f7-8fe244cbc06e None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:00.188419 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08c1cb8-f258-4b78-817f-8b5f6a0611a7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de934d0105bd4d01ba6c06a4a0d36704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:00.189100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c1cb8-f258-4b78-817f-8b5f6a0611a7 service placement] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:00.190085 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c1cb8-f258-4b78-817f-8b5f6a0611a7 service placement] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:00.190085 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c1cb8-f258-4b78-817f-8b5f6a0611a7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:00.192453 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a08c1cb8-f258-4b78-817f-8b5f6a0611a7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:00.208056 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08c1cb8-f258-4b78-817f-8b5f6a0611a7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:00.209756 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08c1cb8-f258-4b78-817f-8b5f6a0611a7 service placement] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:00.220354 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1163/2310] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:47:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:47:07.317779 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c03aa7-30f2-44f2-b72a-93d508be9b78 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:07.326655 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c03aa7-30f2-44f2-b72a-93d508be9b78 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:07.330622 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c007858-cdc8-4a89-8ea6-184dc82853be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:07.331353 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c007858-cdc8-4a89-8ea6-184dc82853be service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:07.331627 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c007858-cdc8-4a89-8ea6-184dc82853be service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:07.332081 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c007858-cdc8-4a89-8ea6-184dc82853be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:07.333812 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7c007858-cdc8-4a89-8ea6-184dc82853be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:07.344032 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c007858-cdc8-4a89-8ea6-184dc82853be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:07.345445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c007858-cdc8-4a89-8ea6-184dc82853be service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:07.354237 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1148/2311] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:47:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:47:10.216114 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=35 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:47:13.375270 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08c1cb8-f258-4b78-817f-8b5f6a0611a7 service placement] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:13.387438 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08c1cb8-f258-4b78-817f-8b5f6a0611a7 service placement] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:13.393141 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aeac3f-ea5e-4b81-abbe-c8ac29161adc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf14b64bf04b2ba74d2fe1fd41ff71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:13.393969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aeac3f-ea5e-4b81-abbe-c8ac29161adc service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:13.394248 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aeac3f-ea5e-4b81-abbe-c8ac29161adc service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:13.394515 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aeac3f-ea5e-4b81-abbe-c8ac29161adc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:13.396395 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-04aeac3f-ea5e-4b81-abbe-c8ac29161adc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:13.407339 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aeac3f-ea5e-4b81-abbe-c8ac29161adc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:13.408971 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aeac3f-ea5e-4b81-abbe-c8ac29161adc service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:13.419001 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1164/2312] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:47:13 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:47:13.824434 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c007858-cdc8-4a89-8ea6-184dc82853be service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:13.826034 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7c007858-cdc8-4a89-8ea6-184dc82853be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:13.842839 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c007858-cdc8-4a89-8ea6-184dc82853be service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:13.848319 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a915e6-76d5-424d-9f3e-713f97da8699 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf14b64bf04b2ba74d2fe1fd41ff71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:13.848927 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a915e6-76d5-424d-9f3e-713f97da8699 service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:13.849207 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a915e6-76d5-424d-9f3e-713f97da8699 service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:13.849479 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a915e6-76d5-424d-9f3e-713f97da8699 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:13.851307 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-f6a915e6-76d5-424d-9f3e-713f97da8699 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:13.869155 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a915e6-76d5-424d-9f3e-713f97da8699 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:13.870736 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a915e6-76d5-424d-9f3e-713f97da8699 service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:13.881490 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1149/2313] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:47:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:47:14.840861 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aeac3f-ea5e-4b81-abbe-c8ac29161adc service nova] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:14.851834 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aeac3f-ea5e-4b81-abbe-c8ac29161adc service nova] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:14.856627 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4997492-30ec-408d-a23e-bae12d70f6a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:14.857247 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4997492-30ec-408d-a23e-bae12d70f6a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:14.857527 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4997492-30ec-408d-a23e-bae12d70f6a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:14.857800 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4997492-30ec-408d-a23e-bae12d70f6a6 admin admin] PATH_INFO: `/v3/users/de27b2da998a4728b07fccfb12294e3d` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:14.858955 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4997492-30ec-408d-a23e-bae12d70f6a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de27b2da998a4728b07fccfb12294e3d)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:14.860887 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4997492-30ec-408d-a23e-bae12d70f6a6 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:14.911690 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-f4997492-30ec-408d-a23e-bae12d70f6a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de27b2da998a4728b07fccfb12294e3d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:14.912569 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-f4997492-30ec-408d-a23e-bae12d70f6a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de27b2da998a4728b07fccfb12294e3d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:14.927448 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-f4997492-30ec-408d-a23e-bae12d70f6a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de27b2da998a4728b07fccfb12294e3d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:14.934907 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-f4997492-30ec-408d-a23e-bae12d70f6a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de27b2da998a4728b07fccfb12294e3d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:14.942042 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-f4997492-30ec-408d-a23e-bae12d70f6a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de27b2da998a4728b07fccfb12294e3d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:14.948560 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1165/2314] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:47:14 2022] DELETE /identity/v3/users/de27b2da998a4728b07fccfb12294e3d => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:47:15.128962 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a915e6-76d5-424d-9f3e-713f97da8699 service nova] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:15.141879 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a915e6-76d5-424d-9f3e-713f97da8699 service nova] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:15.201933 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4aafd-ebc1-4c21-ab45-29c706011c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:15.202565 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4aafd-ebc1-4c21-ab45-29c706011c59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:15.202843 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4aafd-ebc1-4c21-ab45-29c706011c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:15.203112 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4aafd-ebc1-4c21-ab45-29c706011c59 admin admin] PATH_INFO: `/v3/projects/6dcadaaec3664e8dace58c2479d8f534` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:15.204231 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4aafd-ebc1-4c21-ab45-29c706011c59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6dcadaaec3664e8dace58c2479d8f534)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:15.206189 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4aafd-ebc1-4c21-ab45-29c706011c59 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:15.252180 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7fd4aafd-ebc1-4c21-ab45-29c706011c59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6dcadaaec3664e8dace58c2479d8f534', 'request_id': 'req-7fd4aafd-ebc1-4c21-ab45-29c706011c59'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:15.260806 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7fd4aafd-ebc1-4c21-ab45-29c706011c59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6dcadaaec3664e8dace58c2479d8f534', 'request_id': 'req-7fd4aafd-ebc1-4c21-ab45-29c706011c59'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:15.267269 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7fd4aafd-ebc1-4c21-ab45-29c706011c59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6dcadaaec3664e8dace58c2479d8f534', 'request_id': 'req-7fd4aafd-ebc1-4c21-ab45-29c706011c59'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:15.279974 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7fd4aafd-ebc1-4c21-ab45-29c706011c59 admin admin] The token cache is being invalidate because project 6dcadaaec3664e8dace58c2479d8f534 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:47:15.280875 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7fd4aafd-ebc1-4c21-ab45-29c706011c59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:15.281441 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-7fd4aafd-ebc1-4c21-ab45-29c706011c59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:15.283425 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1150/2315] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:47:15 2022] DELETE /identity/v3/projects/6dcadaaec3664e8dace58c2479d8f534 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:47:15.607539 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4997492-30ec-408d-a23e-bae12d70f6a6 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:15.608972 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f4997492-30ec-408d-a23e-bae12d70f6a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:15.664646 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4997492-30ec-408d-a23e-bae12d70f6a6 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:15.711646 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9513b8d-87a3-4ea6-9c2f-b14ac11b9721 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:15.712507 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9513b8d-87a3-4ea6-9c2f-b14ac11b9721 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:15.712813 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9513b8d-87a3-4ea6-9c2f-b14ac11b9721 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:15.713067 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9513b8d-87a3-4ea6-9c2f-b14ac11b9721 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:15.714751 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d9513b8d-87a3-4ea6-9c2f-b14ac11b9721 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:15.745624 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:47:15.784101 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9513b8d-87a3-4ea6-9c2f-b14ac11b9721 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:15.785587 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9513b8d-87a3-4ea6-9c2f-b14ac11b9721 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:15.795751 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1166/2316] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:47:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:47:17.281349 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4aafd-ebc1-4c21-ab45-29c706011c59 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:17.283114 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7fd4aafd-ebc1-4c21-ab45-29c706011c59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:17.341708 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:47:17.370108 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4aafd-ebc1-4c21-ab45-29c706011c59 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:17.376101 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe52a102-8658-4b6d-a7c2-84ebf1fa072e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf14b64bf04b2ba74d2fe1fd41ff71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:17.376595 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52a102-8658-4b6d-a7c2-84ebf1fa072e service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:17.376973 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52a102-8658-4b6d-a7c2-84ebf1fa072e service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:17.377342 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52a102-8658-4b6d-a7c2-84ebf1fa072e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:17.379249 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-fe52a102-8658-4b6d-a7c2-84ebf1fa072e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:17.393520 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe52a102-8658-4b6d-a7c2-84ebf1fa072e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:17.396559 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe52a102-8658-4b6d-a7c2-84ebf1fa072e service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:17.408967 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1151/2317] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:47:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:47:19.074268 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9513b8d-87a3-4ea6-9c2f-b14ac11b9721 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:19.076590 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d9513b8d-87a3-4ea6-9c2f-b14ac11b9721 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:19.158067 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe52a102-8658-4b6d-a7c2-84ebf1fa072e service nova] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:19.159514 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-fe52a102-8658-4b6d-a7c2-84ebf1fa072e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:19.184926 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9513b8d-87a3-4ea6-9c2f-b14ac11b9721 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:19.185183 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe52a102-8658-4b6d-a7c2-84ebf1fa072e service nova] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:19.190073 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2849079-aeaf-4862-9183-8412f03b5250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:19.190221 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d40f6-4386-4bd2-97ef-9e6e809dd74b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de934d0105bd4d01ba6c06a4a0d36704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:19.190711 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d40f6-4386-4bd2-97ef-9e6e809dd74b service placement] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:19.191145 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d40f6-4386-4bd2-97ef-9e6e809dd74b service placement] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:19.191360 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2849079-aeaf-4862-9183-8412f03b5250 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:19.191493 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d40f6-4386-4bd2-97ef-9e6e809dd74b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:19.191925 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2849079-aeaf-4862-9183-8412f03b5250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:19.192196 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2849079-aeaf-4862-9183-8412f03b5250 admin admin] PATH_INFO: `/v3/users/36515ae6f91748babdcf6a9a72f59ed1` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:19.193362 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-337d40f6-4386-4bd2-97ef-9e6e809dd74b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:19.193960 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2849079-aeaf-4862-9183-8412f03b5250 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36515ae6f91748babdcf6a9a72f59ed1)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:19.196802 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2849079-aeaf-4862-9183-8412f03b5250 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:19.207484 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337d40f6-4386-4bd2-97ef-9e6e809dd74b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:19.209085 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337d40f6-4386-4bd2-97ef-9e6e809dd74b service placement] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:19.220928 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1167/2318] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:47:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:47:19.263730 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e2849079-aeaf-4862-9183-8412f03b5250 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36515ae6f91748babdcf6a9a72f59ed1'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:19.271632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e2849079-aeaf-4862-9183-8412f03b5250 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36515ae6f91748babdcf6a9a72f59ed1'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:19.272330 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e2849079-aeaf-4862-9183-8412f03b5250 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36515ae6f91748babdcf6a9a72f59ed1'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:19.280010 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e2849079-aeaf-4862-9183-8412f03b5250 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36515ae6f91748babdcf6a9a72f59ed1'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:19.280758 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e2849079-aeaf-4862-9183-8412f03b5250 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36515ae6f91748babdcf6a9a72f59ed1'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:19.302660 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1152/2319] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:47:19 2022] DELETE /identity/v3/users/36515ae6f91748babdcf6a9a72f59ed1 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:47:19.479173 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d40f6-4386-4bd2-97ef-9e6e809dd74b service placement] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:19.490439 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d40f6-4386-4bd2-97ef-9e6e809dd74b service placement] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:19.548393 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc3645-8e3c-4b88-b9b7-0d74e7f6c702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:19.548967 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc3645-8e3c-4b88-b9b7-0d74e7f6c702 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:19.549247 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc3645-8e3c-4b88-b9b7-0d74e7f6c702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:19.549519 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc3645-8e3c-4b88-b9b7-0d74e7f6c702 admin admin] PATH_INFO: `/v3/projects/824ddb9f6f344a2684cf31d08427306c` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:19.550939 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fc3645-8e3c-4b88-b9b7-0d74e7f6c702 admin admin] RBAC: Authorizing `identity:delete_project(project_id=824ddb9f6f344a2684cf31d08427306c)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:19.552841 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fc3645-8e3c-4b88-b9b7-0d74e7f6c702 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:19.597707 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-03fc3645-8e3c-4b88-b9b7-0d74e7f6c702 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '824ddb9f6f344a2684cf31d08427306c', 'request_id': 'req-03fc3645-8e3c-4b88-b9b7-0d74e7f6c702'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:19.604567 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-03fc3645-8e3c-4b88-b9b7-0d74e7f6c702 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '824ddb9f6f344a2684cf31d08427306c', 'request_id': 'req-03fc3645-8e3c-4b88-b9b7-0d74e7f6c702'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:19.607048 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:47:19.611941 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-03fc3645-8e3c-4b88-b9b7-0d74e7f6c702 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '824ddb9f6f344a2684cf31d08427306c', 'request_id': 'req-03fc3645-8e3c-4b88-b9b7-0d74e7f6c702'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:19.623446 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-03fc3645-8e3c-4b88-b9b7-0d74e7f6c702 admin admin] The token cache is being invalidate because project 824ddb9f6f344a2684cf31d08427306c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:47:19.623732 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-03fc3645-8e3c-4b88-b9b7-0d74e7f6c702 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:19.624345 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-03fc3645-8e3c-4b88-b9b7-0d74e7f6c702 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:19.625798 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1168/2320] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:47:19 2022] DELETE /identity/v3/projects/824ddb9f6f344a2684cf31d08427306c => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:47:19.636128 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10125c-5587-4f1d-ae80-d64bf4a8eea0 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:19.636414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10125c-5587-4f1d-ae80-d64bf4a8eea0 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:19.636690 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10125c-5587-4f1d-ae80-d64bf4a8eea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:19.656428 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-6a10125c-5587-4f1d-ae80-d64bf4a8eea0 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:47:19.657614 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-6a10125c-5587-4f1d-ae80-d64bf4a8eea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:19.714485 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1153/2321] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:47:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:47:19.722019 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22461e7c-fe99-47ca-9428-0a0a0cf38bbb None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:19.722370 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22461e7c-fe99-47ca-9428-0a0a0cf38bbb None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:19.722570 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22461e7c-fe99-47ca-9428-0a0a0cf38bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:19.742961 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-22461e7c-fe99-47ca-9428-0a0a0cf38bbb None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:47:19.744135 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-22461e7c-fe99-47ca-9428-0a0a0cf38bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:19.751874 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1169/2322] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:47:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:47:19.755927 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a10125c-5587-4f1d-ae80-d64bf4a8eea0 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:19.765813 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a10125c-5587-4f1d-ae80-d64bf4a8eea0 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:19.770323 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebe3383-d2ff-4f73-8918-a30e31da7964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:19.770910 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe3383-d2ff-4f73-8918-a30e31da7964 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:19.771184 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe3383-d2ff-4f73-8918-a30e31da7964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:19.771449 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebe3383-d2ff-4f73-8918-a30e31da7964 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:19.772083 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebe3383-d2ff-4f73-8918-a30e31da7964 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:47:19.772377 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebe3383-d2ff-4f73-8918-a30e31da7964 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:19.774508 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebe3383-d2ff-4f73-8918-a30e31da7964 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:19.785760 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1154/2323] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:47:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:47:19.790035 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22461e7c-fe99-47ca-9428-0a0a0cf38bbb None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:19.799221 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22461e7c-fe99-47ca-9428-0a0a0cf38bbb None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:19.803785 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba2df55-23db-44b0-9095-185ca3081d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:19.804379 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba2df55-23db-44b0-9095-185ca3081d32 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:19.804660 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba2df55-23db-44b0-9095-185ca3081d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:19.804930 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba2df55-23db-44b0-9095-185ca3081d32 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:19.805587 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba2df55-23db-44b0-9095-185ca3081d32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:19.807502 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba2df55-23db-44b0-9095-185ca3081d32 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:19.834589 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1170/2324] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:47:19 2022] POST /identity/v3/projects => generated 403 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:47:19.839394 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebe3383-d2ff-4f73-8918-a30e31da7964 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:19.849103 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebe3383-d2ff-4f73-8918-a30e31da7964 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:19.880910 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:47:19.907412 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94edbec-45f4-4167-bed9-c298b1981bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:19.908009 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94edbec-45f4-4167-bed9-c298b1981bfb admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:19.908289 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94edbec-45f4-4167-bed9-c298b1981bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:19.908561 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94edbec-45f4-4167-bed9-c298b1981bfb admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:19.909179 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94edbec-45f4-4167-bed9-c298b1981bfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:19.911111 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94edbec-45f4-4167-bed9-c298b1981bfb admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:19.931009 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1155/2325] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:47:19 2022] POST /identity/v3/users => generated 302 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:47:19.935065 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba2df55-23db-44b0-9095-185ca3081d32 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:19.945673 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba2df55-23db-44b0-9095-185ca3081d32 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:19.950264 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da8b42-d184-4511-bace-cab635b926b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:19.950862 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da8b42-d184-4511-bace-cab635b926b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:19.951145 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da8b42-d184-4511-bace-cab635b926b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:19.951420 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da8b42-d184-4511-bace-cab635b926b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:19.952056 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68da8b42-d184-4511-bace-cab635b926b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:19.953638 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68da8b42-d184-4511-bace-cab635b926b4 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:19.964467 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1171/2326] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:47:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:47:19.967583 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94edbec-45f4-4167-bed9-c298b1981bfb admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:19.977420 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94edbec-45f4-4167-bed9-c298b1981bfb admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:19.982022 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4908f348-c0a5-486f-ad35-770620fc2cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:19.982605 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4908f348-c0a5-486f-ad35-770620fc2cee admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:19.982886 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4908f348-c0a5-486f-ad35-770620fc2cee admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:19.983154 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4908f348-c0a5-486f-ad35-770620fc2cee admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:19.983782 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4908f348-c0a5-486f-ad35-770620fc2cee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:19.985338 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4908f348-c0a5-486f-ad35-770620fc2cee admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:19.995362 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1156/2327] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:47:19 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:47:19.999304 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da8b42-d184-4511-bace-cab635b926b4 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:20.009549 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da8b42-d184-4511-bace-cab635b926b4 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:20.014289 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a53d8af-13f5-4895-9327-e115b0593da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:20.014930 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a53d8af-13f5-4895-9327-e115b0593da2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:20.015211 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a53d8af-13f5-4895-9327-e115b0593da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:20.015482 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a53d8af-13f5-4895-9327-e115b0593da2 admin admin] PATH_INFO: `/v3/projects/664996c5847a4d92916079fc92a42e4d/users/f1ae8b2badaa4f0d88616bb4c864ac3f/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:20.030526 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a53d8af-13f5-4895-9327-e115b0593da2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=664996c5847a4d92916079fc92a42e4d, user_id=f1ae8b2badaa4f0d88616bb4c864ac3f, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:20.033143 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a53d8af-13f5-4895-9327-e115b0593da2 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:20.045507 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1172/2328] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:47:19 2022] PUT /identity/v3/projects/664996c5847a4d92916079fc92a42e4d/users/f1ae8b2badaa4f0d88616bb4c864ac3f/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:47:20.056486 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7023ef3-6f58-4e89-9f2a-f01f1ce2d2a4 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:20.056766 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7023ef3-6f58-4e89-9f2a-f01f1ce2d2a4 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:20.057036 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7023ef3-6f58-4e89-9f2a-f01f1ce2d2a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:20.087317 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-d7023ef3-6f58-4e89-9f2a-f01f1ce2d2a4 None None] MFA Rules not processed for user `f1ae8b2badaa4f0d88616bb4c864ac3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:47:20.088506 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d7023ef3-6f58-4e89-9f2a-f01f1ce2d2a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:20.163968 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1157/2329] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:47:20 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:47:20.168687 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a53d8af-13f5-4895-9327-e115b0593da2 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:20.236508 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a53d8af-13f5-4895-9327-e115b0593da2 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:20.241197 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350264d0-43f9-4e42-9ec8-7be2d4709783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:20.242671 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350264d0-43f9-4e42-9ec8-7be2d4709783 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:20.242952 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350264d0-43f9-4e42-9ec8-7be2d4709783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:20.243228 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350264d0-43f9-4e42-9ec8-7be2d4709783 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:20.243855 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350264d0-43f9-4e42-9ec8-7be2d4709783 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:20.245758 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350264d0-43f9-4e42-9ec8-7be2d4709783 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:20.273245 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1173/2330] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:47:20 2022] POST /identity/v3/projects => generated 405 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:47:20.277433 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7023ef3-6f58-4e89-9f2a-f01f1ce2d2a4 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:20.288237 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7023ef3-6f58-4e89-9f2a-f01f1ce2d2a4 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:20.344042 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941ce64-6694-47ce-a5f7-5f5f5557f769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:20.344626 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941ce64-6694-47ce-a5f7-5f5f5557f769 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:20.344901 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941ce64-6694-47ce-a5f7-5f5f5557f769 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:20.345168 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941ce64-6694-47ce-a5f7-5f5f5557f769 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:20.345799 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6941ce64-6694-47ce-a5f7-5f5f5557f769 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:20.347873 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6941ce64-6694-47ce-a5f7-5f5f5557f769 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:20.369556 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1158/2331] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:47:20 2022] POST /identity/v3/users => generated 309 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:47:20.373592 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350264d0-43f9-4e42-9ec8-7be2d4709783 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:20.385167 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350264d0-43f9-4e42-9ec8-7be2d4709783 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:20.389718 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba17878b-ff37-44df-9291-07c809674c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:20.390351 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17878b-ff37-44df-9291-07c809674c4e admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:20.390635 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17878b-ff37-44df-9291-07c809674c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:20.390908 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17878b-ff37-44df-9291-07c809674c4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:20.391540 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba17878b-ff37-44df-9291-07c809674c4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:20.393126 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba17878b-ff37-44df-9291-07c809674c4e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:20.403683 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1174/2332] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:47:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:47:20.407642 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941ce64-6694-47ce-a5f7-5f5f5557f769 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:20.418653 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941ce64-6694-47ce-a5f7-5f5f5557f769 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:20.423546 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a9558d-73d5-4411-9d5b-2e6104bc6830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:20.424367 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9558d-73d5-4411-9d5b-2e6104bc6830 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:20.424644 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9558d-73d5-4411-9d5b-2e6104bc6830 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:20.424914 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9558d-73d5-4411-9d5b-2e6104bc6830 admin admin] PATH_INFO: `/v3/projects/6be2dc8faf504409a419503a27d5ef4d/users/cc9c3d73176c4a4e9ef9e721c272c92e/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:20.443686 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a9558d-73d5-4411-9d5b-2e6104bc6830 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6be2dc8faf504409a419503a27d5ef4d, user_id=cc9c3d73176c4a4e9ef9e721c272c92e, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:20.449081 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a9558d-73d5-4411-9d5b-2e6104bc6830 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:20.462881 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1159/2333] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:47:20 2022] PUT /identity/v3/projects/6be2dc8faf504409a419503a27d5ef4d/users/cc9c3d73176c4a4e9ef9e721c272c92e/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:47:20.466903 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba17878b-ff37-44df-9291-07c809674c4e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:20.479557 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba17878b-ff37-44df-9291-07c809674c4e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:20.529048 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:47:20.537491 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc93692c-6c5e-407b-b0ac-2955e8d44b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:20.538797 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93692c-6c5e-407b-b0ac-2955e8d44b33 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:20.539261 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93692c-6c5e-407b-b0ac-2955e8d44b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:20.539648 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93692c-6c5e-407b-b0ac-2955e8d44b33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:20.540472 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc93692c-6c5e-407b-b0ac-2955e8d44b33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:20.542120 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc93692c-6c5e-407b-b0ac-2955e8d44b33 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:20.551953 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1175/2334] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:47:20 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:47:20.556733 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a9558d-73d5-4411-9d5b-2e6104bc6830 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:20.566966 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a9558d-73d5-4411-9d5b-2e6104bc6830 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:20.571804 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf05ede-2904-4b8a-beb1-034bf86b4705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:20.572478 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf05ede-2904-4b8a-beb1-034bf86b4705 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:20.572756 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf05ede-2904-4b8a-beb1-034bf86b4705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:20.573026 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf05ede-2904-4b8a-beb1-034bf86b4705 admin admin] PATH_INFO: `/v3/projects/6be2dc8faf504409a419503a27d5ef4d/users/cc9c3d73176c4a4e9ef9e721c272c92e/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:20.574964 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf05ede-2904-4b8a-beb1-034bf86b4705 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6be2dc8faf504409a419503a27d5ef4d, user_id=cc9c3d73176c4a4e9ef9e721c272c92e, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:20.577615 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf05ede-2904-4b8a-beb1-034bf86b4705 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:20.580706 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:47:20.589399 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1160/2335] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:47:20 2022] PUT /identity/v3/projects/6be2dc8faf504409a419503a27d5ef4d/users/cc9c3d73176c4a4e9ef9e721c272c92e/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:47:20.599094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d94d66-8620-496c-b2f3-0063c2f6fc95 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:20.599319 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d94d66-8620-496c-b2f3-0063c2f6fc95 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:20.599594 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d94d66-8620-496c-b2f3-0063c2f6fc95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:20.628634 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-47d94d66-8620-496c-b2f3-0063c2f6fc95 None None] MFA Rules not processed for user `cc9c3d73176c4a4e9ef9e721c272c92e`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:47:20.629890 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-47d94d66-8620-496c-b2f3-0063c2f6fc95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:20.695971 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1176/2336] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:47:20 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:47:20.706441 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf05ede-2904-4b8a-beb1-034bf86b4705 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:20.708124 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-fdf05ede-2904-4b8a-beb1-034bf86b4705 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:20.777947 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf05ede-2904-4b8a-beb1-034bf86b4705 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:20.781570 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8472b5-329e-4e1f-bc87-b4ac40bb14af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:20.782219 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8472b5-329e-4e1f-bc87-b4ac40bb14af service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:20.782494 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8472b5-329e-4e1f-bc87-b4ac40bb14af service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:20.782761 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8472b5-329e-4e1f-bc87-b4ac40bb14af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:20.791878 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8472b5-329e-4e1f-bc87-b4ac40bb14af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:20.793300 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8472b5-329e-4e1f-bc87-b4ac40bb14af service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:20.802413 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1161/2337] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:47:20 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:47:20.939891 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d94d66-8620-496c-b2f3-0063c2f6fc95 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:21.003492 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d94d66-8620-496c-b2f3-0063c2f6fc95 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:21.007997 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2782c77b-da92-418d-9aa3-df7211578919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:21.008631 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2782c77b-da92-418d-9aa3-df7211578919 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:21.008910 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2782c77b-da92-418d-9aa3-df7211578919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:21.009183 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2782c77b-da92-418d-9aa3-df7211578919 admin admin] PATH_INFO: `/v3/users/f1ae8b2badaa4f0d88616bb4c864ac3f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:21.010366 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2782c77b-da92-418d-9aa3-df7211578919 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1ae8b2badaa4f0d88616bb4c864ac3f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:21.012275 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2782c77b-da92-418d-9aa3-df7211578919 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:21.058310 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2782c77b-da92-418d-9aa3-df7211578919 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1ae8b2badaa4f0d88616bb4c864ac3f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:21.058958 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2782c77b-da92-418d-9aa3-df7211578919 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1ae8b2badaa4f0d88616bb4c864ac3f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:21.070526 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2782c77b-da92-418d-9aa3-df7211578919 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1ae8b2badaa4f0d88616bb4c864ac3f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:21.078198 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2782c77b-da92-418d-9aa3-df7211578919 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1ae8b2badaa4f0d88616bb4c864ac3f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:21.083496 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:47:21.086070 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2782c77b-da92-418d-9aa3-df7211578919 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1ae8b2badaa4f0d88616bb4c864ac3f'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:21.093646 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1177/2338] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:47:20 2022] DELETE /identity/v3/users/f1ae8b2badaa4f0d88616bb4c864ac3f => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:47:21.098106 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60acb2-26e4-4a6d-bce1-33e9f067afb6 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:21.098390 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60acb2-26e4-4a6d-bce1-33e9f067afb6 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:21.098664 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60acb2-26e4-4a6d-bce1-33e9f067afb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:21.118838 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-6b60acb2-26e4-4a6d-bce1-33e9f067afb6 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:47:21.119957 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-6b60acb2-26e4-4a6d-bce1-33e9f067afb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:21.300704 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1162/2339] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:47:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:47:21.311038 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2782c77b-da92-418d-9aa3-df7211578919 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:21.312473 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2782c77b-da92-418d-9aa3-df7211578919 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:21.370198 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2782c77b-da92-418d-9aa3-df7211578919 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:21.373669 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb9b91f-8461-4a7d-baaa-96c587425c62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:21.374307 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb9b91f-8461-4a7d-baaa-96c587425c62 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:21.374586 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb9b91f-8461-4a7d-baaa-96c587425c62 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:21.374871 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb9b91f-8461-4a7d-baaa-96c587425c62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:21.383868 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb9b91f-8461-4a7d-baaa-96c587425c62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:21.385280 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb9b91f-8461-4a7d-baaa-96c587425c62 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:21.394915 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1178/2340] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:47:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:47:21.658236 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b60acb2-26e4-4a6d-bce1-33e9f067afb6 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:21.659658 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-6b60acb2-26e4-4a6d-bce1-33e9f067afb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:21.671760 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b60acb2-26e4-4a6d-bce1-33e9f067afb6 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:21.676295 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c073fe66-039b-40b3-aa9f-33d7bb99aeb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:21.676910 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c073fe66-039b-40b3-aa9f-33d7bb99aeb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:21.677179 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c073fe66-039b-40b3-aa9f-33d7bb99aeb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:21.677440 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c073fe66-039b-40b3-aa9f-33d7bb99aeb9 admin admin] PATH_INFO: `/v3/projects/664996c5847a4d92916079fc92a42e4d` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:21.678542 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c073fe66-039b-40b3-aa9f-33d7bb99aeb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=664996c5847a4d92916079fc92a42e4d)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:21.680455 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c073fe66-039b-40b3-aa9f-33d7bb99aeb9 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:21.727161 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c073fe66-039b-40b3-aa9f-33d7bb99aeb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '664996c5847a4d92916079fc92a42e4d', 'request_id': 'req-c073fe66-039b-40b3-aa9f-33d7bb99aeb9'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:21.734247 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c073fe66-039b-40b3-aa9f-33d7bb99aeb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '664996c5847a4d92916079fc92a42e4d', 'request_id': 'req-c073fe66-039b-40b3-aa9f-33d7bb99aeb9'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:21.740360 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c073fe66-039b-40b3-aa9f-33d7bb99aeb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '664996c5847a4d92916079fc92a42e4d', 'request_id': 'req-c073fe66-039b-40b3-aa9f-33d7bb99aeb9'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:21.750849 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c073fe66-039b-40b3-aa9f-33d7bb99aeb9 admin admin] The token cache is being invalidate because project 664996c5847a4d92916079fc92a42e4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:47:21.751130 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c073fe66-039b-40b3-aa9f-33d7bb99aeb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:21.751742 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-c073fe66-039b-40b3-aa9f-33d7bb99aeb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:21.753258 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1163/2341] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:47:21 2022] DELETE /identity/v3/projects/664996c5847a4d92916079fc92a42e4d => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:47:21.757299 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb9b91f-8461-4a7d-baaa-96c587425c62 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:21.766770 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb9b91f-8461-4a7d-baaa-96c587425c62 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:21.823028 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a107b-6748-41be-ac64-b0841f37dd2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:21.823647 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a107b-6748-41be-ac64-b0841f37dd2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:21.823927 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a107b-6748-41be-ac64-b0841f37dd2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:21.824199 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a107b-6748-41be-ac64-b0841f37dd2e admin admin] PATH_INFO: `/v3/users/cc9c3d73176c4a4e9ef9e721c272c92e` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:21.825325 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970a107b-6748-41be-ac64-b0841f37dd2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc9c3d73176c4a4e9ef9e721c272c92e)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:21.827375 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970a107b-6748-41be-ac64-b0841f37dd2e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:21.873039 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-970a107b-6748-41be-ac64-b0841f37dd2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc9c3d73176c4a4e9ef9e721c272c92e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:21.873676 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-970a107b-6748-41be-ac64-b0841f37dd2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc9c3d73176c4a4e9ef9e721c272c92e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:21.885121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-970a107b-6748-41be-ac64-b0841f37dd2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc9c3d73176c4a4e9ef9e721c272c92e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:21.892136 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-970a107b-6748-41be-ac64-b0841f37dd2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc9c3d73176c4a4e9ef9e721c272c92e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:21.899325 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-970a107b-6748-41be-ac64-b0841f37dd2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc9c3d73176c4a4e9ef9e721c272c92e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:21.905089 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1179/2342] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:47:21 2022] DELETE /identity/v3/users/cc9c3d73176c4a4e9ef9e721c272c92e => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:47:22.178478 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c073fe66-039b-40b3-aa9f-33d7bb99aeb9 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:22.179959 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c073fe66-039b-40b3-aa9f-33d7bb99aeb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:22.193126 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c073fe66-039b-40b3-aa9f-33d7bb99aeb9 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:22.215585 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:47:22.255894 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef468b50-a70a-4266-b04a-a88d29f1fe1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:22.256511 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef468b50-a70a-4266-b04a-a88d29f1fe1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:22.256784 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef468b50-a70a-4266-b04a-a88d29f1fe1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:22.259694 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef468b50-a70a-4266-b04a-a88d29f1fe1e admin admin] PATH_INFO: `/v3/projects/6be2dc8faf504409a419503a27d5ef4d` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:22.260895 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef468b50-a70a-4266-b04a-a88d29f1fe1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6be2dc8faf504409a419503a27d5ef4d)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:22.262786 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef468b50-a70a-4266-b04a-a88d29f1fe1e admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:22.308472 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ef468b50-a70a-4266-b04a-a88d29f1fe1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6be2dc8faf504409a419503a27d5ef4d', 'request_id': 'req-ef468b50-a70a-4266-b04a-a88d29f1fe1e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:22.315603 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ef468b50-a70a-4266-b04a-a88d29f1fe1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6be2dc8faf504409a419503a27d5ef4d', 'request_id': 'req-ef468b50-a70a-4266-b04a-a88d29f1fe1e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:22.322644 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ef468b50-a70a-4266-b04a-a88d29f1fe1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6be2dc8faf504409a419503a27d5ef4d', 'request_id': 'req-ef468b50-a70a-4266-b04a-a88d29f1fe1e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:22.332827 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ef468b50-a70a-4266-b04a-a88d29f1fe1e admin admin] The token cache is being invalidate because project 6be2dc8faf504409a419503a27d5ef4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:47:22.333114 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ef468b50-a70a-4266-b04a-a88d29f1fe1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:22.333747 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ef468b50-a70a-4266-b04a-a88d29f1fe1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:22.335829 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1164/2343] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:47:22 2022] DELETE /identity/v3/projects/6be2dc8faf504409a419503a27d5ef4d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:47:22.340869 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876d4c3-bf1d-4b8a-9bef-08cd2441f1ef None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:22.341318 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876d4c3-bf1d-4b8a-9bef-08cd2441f1ef None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:22.341431 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876d4c3-bf1d-4b8a-9bef-08cd2441f1ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:22.361932 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-4876d4c3-bf1d-4b8a-9bef-08cd2441f1ef None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:47:22.363129 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4876d4c3-bf1d-4b8a-9bef-08cd2441f1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:22.375142 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:47:22.423541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1180/2344] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:47:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:47:22.431827 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803bff11-0a68-4696-a340-b405d95647aa None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:22.432107 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803bff11-0a68-4696-a340-b405d95647aa None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:22.432371 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803bff11-0a68-4696-a340-b405d95647aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:22.452278 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-803bff11-0a68-4696-a340-b405d95647aa None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:47:22.453489 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-803bff11-0a68-4696-a340-b405d95647aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:22.461746 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1165/2345] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:47:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:47:22.466108 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876d4c3-bf1d-4b8a-9bef-08cd2441f1ef None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:22.475567 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876d4c3-bf1d-4b8a-9bef-08cd2441f1ef None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:22.480174 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f4f7d2-ba65-46a5-892d-9dec63b2edae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:22.480775 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f4f7d2-ba65-46a5-892d-9dec63b2edae admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:22.481054 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f4f7d2-ba65-46a5-892d-9dec63b2edae admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:22.481330 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f4f7d2-ba65-46a5-892d-9dec63b2edae admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:22.481995 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f4f7d2-ba65-46a5-892d-9dec63b2edae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:47:22.482303 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f4f7d2-ba65-46a5-892d-9dec63b2edae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:22.483890 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f4f7d2-ba65-46a5-892d-9dec63b2edae admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:22.496596 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1181/2346] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:47:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:47:22.500867 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803bff11-0a68-4696-a340-b405d95647aa None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:22.510799 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803bff11-0a68-4696-a340-b405d95647aa None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:22.515385 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99bae0-674f-457b-8a3d-1b13e4ebec3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:22.515973 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99bae0-674f-457b-8a3d-1b13e4ebec3b admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:22.516535 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99bae0-674f-457b-8a3d-1b13e4ebec3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:22.516802 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99bae0-674f-457b-8a3d-1b13e4ebec3b admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:22.517426 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e99bae0-674f-457b-8a3d-1b13e4ebec3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:22.519321 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e99bae0-674f-457b-8a3d-1b13e4ebec3b admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:22.547351 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1166/2347] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:47:22 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:47:22.551644 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f4f7d2-ba65-46a5-892d-9dec63b2edae admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:22.562312 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f4f7d2-ba65-46a5-892d-9dec63b2edae admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:22.620359 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca02ef3-c49f-417a-ba63-06dc8846cafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:22.620986 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca02ef3-c49f-417a-ba63-06dc8846cafc admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:22.621265 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca02ef3-c49f-417a-ba63-06dc8846cafc admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:22.621535 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca02ef3-c49f-417a-ba63-06dc8846cafc admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:22.622202 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca02ef3-c49f-417a-ba63-06dc8846cafc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:22.624112 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca02ef3-c49f-417a-ba63-06dc8846cafc admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:22.645497 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1182/2348] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:47:22 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:47:22.649680 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99bae0-674f-457b-8a3d-1b13e4ebec3b admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:22.659695 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99bae0-674f-457b-8a3d-1b13e4ebec3b admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:22.664234 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21484d79-59b7-4e8b-8f62-5be8dc353bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:22.664821 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21484d79-59b7-4e8b-8f62-5be8dc353bfe admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:22.665099 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21484d79-59b7-4e8b-8f62-5be8dc353bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:22.665368 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21484d79-59b7-4e8b-8f62-5be8dc353bfe admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:22.666044 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21484d79-59b7-4e8b-8f62-5be8dc353bfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:22.667650 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21484d79-59b7-4e8b-8f62-5be8dc353bfe admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:22.677375 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1167/2349] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:47:22 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:47:22.681730 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca02ef3-c49f-417a-ba63-06dc8846cafc admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:22.691479 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca02ef3-c49f-417a-ba63-06dc8846cafc admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:22.696713 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda122fe-38fc-41b7-a214-a47a6dfec127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:22.697295 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda122fe-38fc-41b7-a214-a47a6dfec127 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:22.697583 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda122fe-38fc-41b7-a214-a47a6dfec127 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:22.697879 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda122fe-38fc-41b7-a214-a47a6dfec127 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:22.698549 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda122fe-38fc-41b7-a214-a47a6dfec127 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:22.700121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda122fe-38fc-41b7-a214-a47a6dfec127 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:22.710516 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1183/2350] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:47:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:47:22.714347 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21484d79-59b7-4e8b-8f62-5be8dc353bfe admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:22.723981 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21484d79-59b7-4e8b-8f62-5be8dc353bfe admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:22.728498 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc0dce3-8ad6-4a64-8f7d-ce46433c5979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:22.729124 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc0dce3-8ad6-4a64-8f7d-ce46433c5979 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:22.729397 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc0dce3-8ad6-4a64-8f7d-ce46433c5979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:22.729663 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc0dce3-8ad6-4a64-8f7d-ce46433c5979 admin admin] PATH_INFO: `/v3/projects/a37e7dd2fe3f474e82fa9b42072fd7bc/users/76640fe329d24af9b39235f60209b133/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:22.744780 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc0dce3-8ad6-4a64-8f7d-ce46433c5979 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a37e7dd2fe3f474e82fa9b42072fd7bc, user_id=76640fe329d24af9b39235f60209b133, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:22.747453 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc0dce3-8ad6-4a64-8f7d-ce46433c5979 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:22.760621 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1168/2351] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:47:22 2022] PUT /identity/v3/projects/a37e7dd2fe3f474e82fa9b42072fd7bc/users/76640fe329d24af9b39235f60209b133/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:47:22.771783 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e23e00b-69d3-4efe-8e1c-699ed0385cab None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:22.772100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e23e00b-69d3-4efe-8e1c-699ed0385cab None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:22.772379 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e23e00b-69d3-4efe-8e1c-699ed0385cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:22.800569 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-3e23e00b-69d3-4efe-8e1c-699ed0385cab None None] MFA Rules not processed for user `76640fe329d24af9b39235f60209b133`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:47:22.801782 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3e23e00b-69d3-4efe-8e1c-699ed0385cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:22.865282 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1184/2352] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:47:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:47:22.879327 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc0dce3-8ad6-4a64-8f7d-ce46433c5979 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:22.880949 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-0bc0dce3-8ad6-4a64-8f7d-ce46433c5979 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:22.933339 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:47:22.937717 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc0dce3-8ad6-4a64-8f7d-ce46433c5979 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:22.942747 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f7e861-602d-4fd6-abb4-5a368bedca4c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6a46bbd249c4434ad7d9442754f098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:22.943544 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f7e861-602d-4fd6-abb4-5a368bedca4c service glance] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:22.944054 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f7e861-602d-4fd6-abb4-5a368bedca4c service glance] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:22.944514 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f7e861-602d-4fd6-abb4-5a368bedca4c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:22.953000 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f7e861-602d-4fd6-abb4-5a368bedca4c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:22.954454 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f7e861-602d-4fd6-abb4-5a368bedca4c service glance] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:22.963098 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1169/2353] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:47:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:47:32.862369 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=91,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:47:32.959106 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:47:39.588702 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e23e00b-69d3-4efe-8e1c-699ed0385cab None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:39.664077 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e23e00b-69d3-4efe-8e1c-699ed0385cab None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:39.669033 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7475d-b612-4498-a04a-92fb0744de09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:39.669667 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7475d-b612-4498-a04a-92fb0744de09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:39.670106 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7475d-b612-4498-a04a-92fb0744de09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:39.670288 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7475d-b612-4498-a04a-92fb0744de09 admin admin] PATH_INFO: `/v3/users/76640fe329d24af9b39235f60209b133` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:39.671401 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f7475d-b612-4498-a04a-92fb0744de09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76640fe329d24af9b39235f60209b133)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:39.673313 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f7475d-b612-4498-a04a-92fb0744de09 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:39.728460 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e0f7475d-b612-4498-a04a-92fb0744de09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76640fe329d24af9b39235f60209b133'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:39.729244 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e0f7475d-b612-4498-a04a-92fb0744de09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76640fe329d24af9b39235f60209b133'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:39.743859 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e0f7475d-b612-4498-a04a-92fb0744de09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76640fe329d24af9b39235f60209b133'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:39.760045 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e0f7475d-b612-4498-a04a-92fb0744de09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76640fe329d24af9b39235f60209b133'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:39.772824 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e0f7475d-b612-4498-a04a-92fb0744de09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76640fe329d24af9b39235f60209b133'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:39.780286 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1185/2354] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:47:39 2022] DELETE /identity/v3/users/76640fe329d24af9b39235f60209b133 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:47:39.784502 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb51d0-d983-450e-9cbf-0effc15dc308 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:39.784951 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb51d0-d983-450e-9cbf-0effc15dc308 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:39.785204 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb51d0-d983-450e-9cbf-0effc15dc308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:39.805750 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-03bb51d0-d983-450e-9cbf-0effc15dc308 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:47:39.807348 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-03bb51d0-d983-450e-9cbf-0effc15dc308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:39.886840 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1170/2355] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:47:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:47:39.897489 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7475d-b612-4498-a04a-92fb0744de09 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:39.899209 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e0f7475d-b612-4498-a04a-92fb0744de09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:39.959644 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7475d-b612-4498-a04a-92fb0744de09 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:39.963234 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca99d32a-3cc3-47cf-a80e-85eca9e0253d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:39.963844 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca99d32a-3cc3-47cf-a80e-85eca9e0253d service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:39.964152 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca99d32a-3cc3-47cf-a80e-85eca9e0253d service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:39.964408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca99d32a-3cc3-47cf-a80e-85eca9e0253d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:39.973651 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca99d32a-3cc3-47cf-a80e-85eca9e0253d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:39.975152 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca99d32a-3cc3-47cf-a80e-85eca9e0253d service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:39.985338 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1186/2356] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:47:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:47:40.263606 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bb51d0-d983-450e-9cbf-0effc15dc308 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:40.265009 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-03bb51d0-d983-450e-9cbf-0effc15dc308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:40.278704 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bb51d0-d983-450e-9cbf-0effc15dc308 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:40.284048 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1732da73-5c99-4de9-a72f-fdd1fe709a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:40.284779 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1732da73-5c99-4de9-a72f-fdd1fe709a76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:40.285124 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1732da73-5c99-4de9-a72f-fdd1fe709a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:40.285458 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1732da73-5c99-4de9-a72f-fdd1fe709a76 admin admin] PATH_INFO: `/v3/projects/a37e7dd2fe3f474e82fa9b42072fd7bc` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:40.286668 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1732da73-5c99-4de9-a72f-fdd1fe709a76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a37e7dd2fe3f474e82fa9b42072fd7bc)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:40.289237 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1732da73-5c99-4de9-a72f-fdd1fe709a76 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:40.336688 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1732da73-5c99-4de9-a72f-fdd1fe709a76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a37e7dd2fe3f474e82fa9b42072fd7bc', 'request_id': 'req-1732da73-5c99-4de9-a72f-fdd1fe709a76'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:40.343377 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1732da73-5c99-4de9-a72f-fdd1fe709a76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a37e7dd2fe3f474e82fa9b42072fd7bc', 'request_id': 'req-1732da73-5c99-4de9-a72f-fdd1fe709a76'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:40.349754 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1732da73-5c99-4de9-a72f-fdd1fe709a76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a37e7dd2fe3f474e82fa9b42072fd7bc', 'request_id': 'req-1732da73-5c99-4de9-a72f-fdd1fe709a76'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:40.360417 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1732da73-5c99-4de9-a72f-fdd1fe709a76 admin admin] The token cache is being invalidate because project a37e7dd2fe3f474e82fa9b42072fd7bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:47:40.360705 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1732da73-5c99-4de9-a72f-fdd1fe709a76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:40.361353 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-1732da73-5c99-4de9-a72f-fdd1fe709a76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:47:40.362855 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1171/2357] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:47:40 2022] DELETE /identity/v3/projects/a37e7dd2fe3f474e82fa9b42072fd7bc => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:47:40.368571 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca258ae6-e5d1-404c-a3bb-d1a052125e56 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:40.368863 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca258ae6-e5d1-404c-a3bb-d1a052125e56 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:40.369149 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca258ae6-e5d1-404c-a3bb-d1a052125e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:40.389319 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-ca258ae6-e5d1-404c-a3bb-d1a052125e56 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:47:40.390533 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ca258ae6-e5d1-404c-a3bb-d1a052125e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:40.408221 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:47:40.453715 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1187/2358] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:47:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:47:40.461788 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa49b431-7157-4ef6-926b-2f810d0b9cc2 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:40.462101 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa49b431-7157-4ef6-926b-2f810d0b9cc2 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:40.462378 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa49b431-7157-4ef6-926b-2f810d0b9cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:40.483978 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-aa49b431-7157-4ef6-926b-2f810d0b9cc2 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:47:40.485083 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-aa49b431-7157-4ef6-926b-2f810d0b9cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:40.493233 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1172/2359] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:47:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:47:40.497745 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca258ae6-e5d1-404c-a3bb-d1a052125e56 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:40.507297 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca258ae6-e5d1-404c-a3bb-d1a052125e56 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:40.511937 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884c7705-a2e1-4ef6-9e47-071626a3e39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:40.512540 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c7705-a2e1-4ef6-9e47-071626a3e39d admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:40.512826 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c7705-a2e1-4ef6-9e47-071626a3e39d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:40.513102 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c7705-a2e1-4ef6-9e47-071626a3e39d admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:40.513742 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884c7705-a2e1-4ef6-9e47-071626a3e39d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:47:40.514084 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884c7705-a2e1-4ef6-9e47-071626a3e39d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:40.515689 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884c7705-a2e1-4ef6-9e47-071626a3e39d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:40.528206 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1188/2360] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:47:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:47:40.532561 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa49b431-7157-4ef6-926b-2f810d0b9cc2 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:40.543203 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa49b431-7157-4ef6-926b-2f810d0b9cc2 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:40.548149 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a355fe-f85b-4e7c-bfce-02285944ba59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:40.548756 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a355fe-f85b-4e7c-bfce-02285944ba59 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:40.549033 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a355fe-f85b-4e7c-bfce-02285944ba59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:40.549301 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a355fe-f85b-4e7c-bfce-02285944ba59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:40.549979 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a355fe-f85b-4e7c-bfce-02285944ba59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:40.551875 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a355fe-f85b-4e7c-bfce-02285944ba59 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:40.580478 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1173/2361] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:47:40 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:47:40.584644 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884c7705-a2e1-4ef6-9e47-071626a3e39d admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:40.595097 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884c7705-a2e1-4ef6-9e47-071626a3e39d admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:40.658345 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6ce9fd-3216-4e09-99ab-91031d76ea3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:40.658964 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ce9fd-3216-4e09-99ab-91031d76ea3b admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:40.659247 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ce9fd-3216-4e09-99ab-91031d76ea3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:40.659526 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ce9fd-3216-4e09-99ab-91031d76ea3b admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:40.660154 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6ce9fd-3216-4e09-99ab-91031d76ea3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:40.662133 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6ce9fd-3216-4e09-99ab-91031d76ea3b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:40.687917 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1189/2362] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:47:40 2022] POST /identity/v3/users => generated 285 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:47:40.689445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a355fe-f85b-4e7c-bfce-02285944ba59 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:40.700475 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a355fe-f85b-4e7c-bfce-02285944ba59 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:40.706838 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e463b0-1331-412c-bd8d-365a9edb471d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:40.707481 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e463b0-1331-412c-bd8d-365a9edb471d admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:40.707758 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e463b0-1331-412c-bd8d-365a9edb471d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:40.708027 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e463b0-1331-412c-bd8d-365a9edb471d admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:40.708731 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e463b0-1331-412c-bd8d-365a9edb471d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:40.710359 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e463b0-1331-412c-bd8d-365a9edb471d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:40.723357 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1174/2363] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:47:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:47:40.728319 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6ce9fd-3216-4e09-99ab-91031d76ea3b admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:40.741963 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6ce9fd-3216-4e09-99ab-91031d76ea3b admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:40.748316 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14194a8-9c5d-4ff3-a094-e40eadc9e4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:40.749218 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14194a8-9c5d-4ff3-a094-e40eadc9e4ba admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:40.749651 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14194a8-9c5d-4ff3-a094-e40eadc9e4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:40.749997 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14194a8-9c5d-4ff3-a094-e40eadc9e4ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:40.750815 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14194a8-9c5d-4ff3-a094-e40eadc9e4ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:40.752619 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14194a8-9c5d-4ff3-a094-e40eadc9e4ba admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:40.770274 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1190/2364] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:47:40 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:47:40.771603 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e463b0-1331-412c-bd8d-365a9edb471d admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:40.782220 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e463b0-1331-412c-bd8d-365a9edb471d admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:40.787035 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fe0e04-4e30-4079-910e-2d64025a90c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:40.787598 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe0e04-4e30-4079-910e-2d64025a90c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:40.787752 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe0e04-4e30-4079-910e-2d64025a90c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:40.788251 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe0e04-4e30-4079-910e-2d64025a90c3 admin admin] PATH_INFO: `/v3/projects/ca6444aebef74b2fb52ee7b66e037393/users/a0803fd2f9934228b11295cdd8d443dd/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:40.804741 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fe0e04-4e30-4079-910e-2d64025a90c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca6444aebef74b2fb52ee7b66e037393, user_id=a0803fd2f9934228b11295cdd8d443dd, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:40.807431 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fe0e04-4e30-4079-910e-2d64025a90c3 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:40.824080 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1175/2365] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:47:40 2022] PUT /identity/v3/projects/ca6444aebef74b2fb52ee7b66e037393/users/a0803fd2f9934228b11295cdd8d443dd/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:47:40.834349 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb8baf-2797-49c3-aca3-049bab02cd64 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:40.834735 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb8baf-2797-49c3-aca3-049bab02cd64 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:40.835084 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb8baf-2797-49c3-aca3-049bab02cd64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:40.896169 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-a3fb8baf-2797-49c3-aca3-049bab02cd64 None None] MFA Rules not processed for user `a0803fd2f9934228b11295cdd8d443dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:47:40.897569 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a3fb8baf-2797-49c3-aca3-049bab02cd64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:40.975319 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1191/2366] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:47:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:47:40.988165 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fe0e04-4e30-4079-910e-2d64025a90c3 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:40.989804 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-15fe0e04-4e30-4079-910e-2d64025a90c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:41.012632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:47:41.054604 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fe0e04-4e30-4079-910e-2d64025a90c3 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:41.058760 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476e98dd-7f37-43f0-8b0b-5f0bbe164267 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6a46bbd249c4434ad7d9442754f098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:41.059392 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e98dd-7f37-43f0-8b0b-5f0bbe164267 service glance] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:41.059665 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e98dd-7f37-43f0-8b0b-5f0bbe164267 service glance] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:41.059927 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e98dd-7f37-43f0-8b0b-5f0bbe164267 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:41.068535 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476e98dd-7f37-43f0-8b0b-5f0bbe164267 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:41.070532 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476e98dd-7f37-43f0-8b0b-5f0bbe164267 service glance] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:41.081578 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1176/2367] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:47:40 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:47:41.085555 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c60bac-40bf-4318-b313-56ab286b60d5 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:41.086375 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c60bac-40bf-4318-b313-56ab286b60d5 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:41.086674 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c60bac-40bf-4318-b313-56ab286b60d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:41.107644 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-71c60bac-40bf-4318-b313-56ab286b60d5 None None] MFA Rules not processed for user `c81e5fb007614cf2992f5e02e85d6307`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:47:41.108896 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-71c60bac-40bf-4318-b313-56ab286b60d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:41.122386 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:47:41.168892 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1192/2368] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Sep 28 15:47:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:47:41.216357 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476e98dd-7f37-43f0-8b0b-5f0bbe164267 service glance] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:41.217747 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-476e98dd-7f37-43f0-8b0b-5f0bbe164267 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:41.288665 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476e98dd-7f37-43f0-8b0b-5f0bbe164267 service glance] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:41.294063 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bd9727-0014-4d03-9951-6a2d41fa7a64 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '328051fe7607464f8f5dc0dcc01a817a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:41.294792 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd9727-0014-4d03-9951-6a2d41fa7a64 service swift] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:41.295027 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd9727-0014-4d03-9951-6a2d41fa7a64 service swift] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:41.295299 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd9727-0014-4d03-9951-6a2d41fa7a64 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:41.303686 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bd9727-0014-4d03-9951-6a2d41fa7a64 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:41.305811 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bd9727-0014-4d03-9951-6a2d41fa7a64 service swift] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:41.314459 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1177/2369] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 15:47:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 15:47:42.751285 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c60bac-40bf-4318-b313-56ab286b60d5 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:42.753146 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-71c60bac-40bf-4318-b313-56ab286b60d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:42.914671 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c60bac-40bf-4318-b313-56ab286b60d5 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:42.920667 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1f6f90-4039-49f6-a81b-dcecbbae8930 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf14b64bf04b2ba74d2fe1fd41ff71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:42.921373 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f6f90-4039-49f6-a81b-dcecbbae8930 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:42.921720 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f6f90-4039-49f6-a81b-dcecbbae8930 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:42.922099 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1f6f90-4039-49f6-a81b-dcecbbae8930 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:42.923921 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ef1f6f90-4039-49f6-a81b-dcecbbae8930 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:42.938255 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1f6f90-4039-49f6-a81b-dcecbbae8930 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:42.939989 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1f6f90-4039-49f6-a81b-dcecbbae8930 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:42.952304 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1193/2370] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:47:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:47:51.311245 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=49 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:47:52.947102 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=51 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:47:57.441287 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e5577-2344-405d-b235-fb0088e9b1a0 None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:57.441840 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e5577-2344-405d-b235-fb0088e9b1a0 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:57.441840 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124e5577-2344-405d-b235-fb0088e9b1a0 None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:57.443576 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1178/2371] 10.222.0.52 () {54 vars in 921 bytes} [Wed Sep 28 15:47:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:47:57.449420 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e8d8d-83bb-4db8-b0be-8dc88aeb6903 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:57.449932 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e8d8d-83bb-4db8-b0be-8dc88aeb6903 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:57.450045 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e8d8d-83bb-4db8-b0be-8dc88aeb6903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:57.473371 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-e92e8d8d-83bb-4db8-b0be-8dc88aeb6903 None None] MFA Rules not processed for user `8daf14b64bf04b2ba74d2fe1fd41ff71`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:47:57.475782 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e92e8d8d-83bb-4db8-b0be-8dc88aeb6903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:57.486213 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1194/2372] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Sep 28 15:47:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:47:57.500989 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124e5577-2344-405d-b235-fb0088e9b1a0 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:57.503235 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-124e5577-2344-405d-b235-fb0088e9b1a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:57.518300 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124e5577-2344-405d-b235-fb0088e9b1a0 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:57.521873 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cbe71b-51d5-4005-bbaa-a600f3ee319f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6a46bbd249c4434ad7d9442754f098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:57.522487 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cbe71b-51d5-4005-bbaa-a600f3ee319f service glance] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:57.522809 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cbe71b-51d5-4005-bbaa-a600f3ee319f service glance] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:57.523082 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cbe71b-51d5-4005-bbaa-a600f3ee319f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:57.534621 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cbe71b-51d5-4005-bbaa-a600f3ee319f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:57.536033 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cbe71b-51d5-4005-bbaa-a600f3ee319f service glance] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:57.548648 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1179/2373] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:47:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:47:57.630000 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab56dd-f000-4c88-b108-e5a1a4bb8564 None None] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:57.630362 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab56dd-f000-4c88-b108-e5a1a4bb8564 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:57.630725 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab56dd-f000-4c88-b108-e5a1a4bb8564 None None] PATH_INFO: `/` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:57.632158 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1195/2374] 10.222.0.52 () {54 vars in 921 bytes} [Wed Sep 28 15:47:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:47:57.637386 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27f310-e75a-4a41-b3b6-1a1dd34b3b6f None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:57.637952 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27f310-e75a-4a41-b3b6-1a1dd34b3b6f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:57.638224 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf27f310-e75a-4a41-b3b6-1a1dd34b3b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:57.658199 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-cf27f310-e75a-4a41-b3b6-1a1dd34b3b6f None None] MFA Rules not processed for user `b13b0533444a449ead00a389c5f6bd1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:47:57.659390 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-cf27f310-e75a-4a41-b3b6-1a1dd34b3b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:57.712900 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1180/2375] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Sep 28 15:47:57 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:47:57.725586 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ab56dd-f000-4c88-b108-e5a1a4bb8564 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:57.728024 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-49ab56dd-f000-4c88-b108-e5a1a4bb8564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:57.741794 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ab56dd-f000-4c88-b108-e5a1a4bb8564 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:57.745745 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48eaaea-dcc0-4cf1-9d93-616c27356bd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:57.746496 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48eaaea-dcc0-4cf1-9d93-616c27356bd4 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:57.748062 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48eaaea-dcc0-4cf1-9d93-616c27356bd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:57.748062 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48eaaea-dcc0-4cf1-9d93-616c27356bd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:57.754964 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48eaaea-dcc0-4cf1-9d93-616c27356bd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:57.756455 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48eaaea-dcc0-4cf1-9d93-616c27356bd4 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:57.769036 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1196/2376] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:47:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:47:57.881692 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285dac0-3b2b-47e5-9944-d9c85458963d None None] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:57.882027 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285dac0-3b2b-47e5-9944-d9c85458963d None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:57.882301 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285dac0-3b2b-47e5-9944-d9c85458963d None None] PATH_INFO: `/` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:57.883649 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1181/2377] 10.222.0.52 () {54 vars in 921 bytes} [Wed Sep 28 15:47:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 28 15:47:57.888350 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2068ff6b-a537-42c1-a2ba-6e08729f9d4d None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:57.888632 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2068ff6b-a537-42c1-a2ba-6e08729f9d4d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:57.888907 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2068ff6b-a537-42c1-a2ba-6e08729f9d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:57.908738 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-2068ff6b-a537-42c1-a2ba-6e08729f9d4d None None] MFA Rules not processed for user `8daf14b64bf04b2ba74d2fe1fd41ff71`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:47:57.909975 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2068ff6b-a537-42c1-a2ba-6e08729f9d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:57.919801 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1197/2378] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Sep 28 15:47:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:47:57.930304 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285dac0-3b2b-47e5-9944-d9c85458963d None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:47:57.931938 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c285dac0-3b2b-47e5-9944-d9c85458963d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:47:57.944858 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285dac0-3b2b-47e5-9944-d9c85458963d None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:47:57.948596 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dea716-7801-4cf5-b71f-7c146fa66b3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:47:57.949182 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dea716-7801-4cf5-b71f-7c146fa66b3c service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:47:57.949460 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dea716-7801-4cf5-b71f-7c146fa66b3c service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:47:57.949729 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dea716-7801-4cf5-b71f-7c146fa66b3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:47:57.960187 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dea716-7801-4cf5-b71f-7c146fa66b3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:47:57.961582 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dea716-7801-4cf5-b71f-7c146fa66b3c service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:47:57.972070 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1182/2379] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:47:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:48:01.678884 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de03ef5-c121-4b0a-a602-963d95faaa29 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:01.679470 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de03ef5-c121-4b0a-a602-963d95faaa29 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:01.679669 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de03ef5-c121-4b0a-a602-963d95faaa29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:01.705370 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-6de03ef5-c121-4b0a-a602-963d95faaa29 None None] MFA Rules not processed for user `c81e5fb007614cf2992f5e02e85d6307`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:01.706926 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6de03ef5-c121-4b0a-a602-963d95faaa29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:01.714478 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1198/2380] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Sep 28 15:48:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:01.728456 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dea716-7801-4cf5-b71f-7c146fa66b3c service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:01.739619 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dea716-7801-4cf5-b71f-7c146fa66b3c service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:01.745522 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bebb79-d96b-4554-ae3a-550a60c0d345 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '328051fe7607464f8f5dc0dcc01a817a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:01.746177 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bebb79-d96b-4554-ae3a-550a60c0d345 service swift] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:01.746447 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bebb79-d96b-4554-ae3a-550a60c0d345 service swift] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:01.746710 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bebb79-d96b-4554-ae3a-550a60c0d345 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:01.755137 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bebb79-d96b-4554-ae3a-550a60c0d345 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:01.756696 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bebb79-d96b-4554-ae3a-550a60c0d345 service swift] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:01.765356 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1183/2381] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 15:48:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 15:48:04.500212 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de03ef5-c121-4b0a-a602-963d95faaa29 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:04.510950 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de03ef5-c121-4b0a-a602-963d95faaa29 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:04.514487 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b21472-3461-49d1-b959-295a4cb2a505 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:04.515092 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b21472-3461-49d1-b959-295a4cb2a505 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:04.515372 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b21472-3461-49d1-b959-295a4cb2a505 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:04.515642 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b21472-3461-49d1-b959-295a4cb2a505 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:04.517412 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c8b21472-3461-49d1-b959-295a4cb2a505 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:04.602343 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b21472-3461-49d1-b959-295a4cb2a505 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:04.603961 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b21472-3461-49d1-b959-295a4cb2a505 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:04.615613 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1199/2382] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:48:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:48:08.564417 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bebb79-d96b-4554-ae3a-550a60c0d345 service swift] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:08.569974 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-b5bebb79-d96b-4554-ae3a-550a60c0d345 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:08.593939 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bebb79-d96b-4554-ae3a-550a60c0d345 service swift] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:08.600705 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b62c7a-98b2-460d-9aad-3520fb8a0c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:08.601663 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b62c7a-98b2-460d-9aad-3520fb8a0c95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:08.602371 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b62c7a-98b2-460d-9aad-3520fb8a0c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:08.602868 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b62c7a-98b2-460d-9aad-3520fb8a0c95 admin admin] PATH_INFO: `/v3/users/634c0af05b334b488f2fa358e574172b` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:08.604729 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b62c7a-98b2-460d-9aad-3520fb8a0c95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=634c0af05b334b488f2fa358e574172b)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:08.607302 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b62c7a-98b2-460d-9aad-3520fb8a0c95 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:08.723737 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-87b62c7a-98b2-460d-9aad-3520fb8a0c95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '634c0af05b334b488f2fa358e574172b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:08.748172 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-87b62c7a-98b2-460d-9aad-3520fb8a0c95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '634c0af05b334b488f2fa358e574172b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:08.749442 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-87b62c7a-98b2-460d-9aad-3520fb8a0c95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '634c0af05b334b488f2fa358e574172b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:08.761070 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-87b62c7a-98b2-460d-9aad-3520fb8a0c95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '634c0af05b334b488f2fa358e574172b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:08.770376 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-87b62c7a-98b2-460d-9aad-3520fb8a0c95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '634c0af05b334b488f2fa358e574172b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:08.799954 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1184/2383] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:48:08 2022] DELETE /identity/v3/users/634c0af05b334b488f2fa358e574172b => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:09.024565 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b21472-3461-49d1-b959-295a4cb2a505 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:09.034981 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b21472-3461-49d1-b959-295a4cb2a505 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:09.107358 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef03b1-91e0-4ad8-88a6-4e50dc5696e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:09.108021 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef03b1-91e0-4ad8-88a6-4e50dc5696e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:09.108302 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef03b1-91e0-4ad8-88a6-4e50dc5696e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:09.108584 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef03b1-91e0-4ad8-88a6-4e50dc5696e6 admin admin] PATH_INFO: `/v3/projects/62692d2b160040879aecaf4d44ad19b3` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:09.109726 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebef03b1-91e0-4ad8-88a6-4e50dc5696e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62692d2b160040879aecaf4d44ad19b3)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:09.111769 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebef03b1-91e0-4ad8-88a6-4e50dc5696e6 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:09.167616 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ebef03b1-91e0-4ad8-88a6-4e50dc5696e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62692d2b160040879aecaf4d44ad19b3', 'request_id': 'req-ebef03b1-91e0-4ad8-88a6-4e50dc5696e6'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:09.174659 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ebef03b1-91e0-4ad8-88a6-4e50dc5696e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62692d2b160040879aecaf4d44ad19b3', 'request_id': 'req-ebef03b1-91e0-4ad8-88a6-4e50dc5696e6'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:09.188775 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ebef03b1-91e0-4ad8-88a6-4e50dc5696e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62692d2b160040879aecaf4d44ad19b3', 'request_id': 'req-ebef03b1-91e0-4ad8-88a6-4e50dc5696e6'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:09.192714 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:09.209027 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ebef03b1-91e0-4ad8-88a6-4e50dc5696e6 admin admin] The token cache is being invalidate because project 62692d2b160040879aecaf4d44ad19b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:48:09.209923 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ebef03b1-91e0-4ad8-88a6-4e50dc5696e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:09.211051 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ebef03b1-91e0-4ad8-88a6-4e50dc5696e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:09.213955 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1200/2384] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:48:09 2022] DELETE /identity/v3/projects/62692d2b160040879aecaf4d44ad19b3 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:09.218763 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c55e1-0b3b-4fb1-85fc-bc0f9a51aa54 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:09.219043 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c55e1-0b3b-4fb1-85fc-bc0f9a51aa54 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:09.219315 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c55e1-0b3b-4fb1-85fc-bc0f9a51aa54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:09.248996 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-420c55e1-0b3b-4fb1-85fc-bc0f9a51aa54 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:09.250318 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-420c55e1-0b3b-4fb1-85fc-bc0f9a51aa54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:09.311617 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:09.341583 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1185/2385] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:09.349472 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0a5a2-4912-4489-a5de-a072a94f900e None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:09.349757 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0a5a2-4912-4489-a5de-a072a94f900e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:09.350133 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0a5a2-4912-4489-a5de-a072a94f900e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:09.375989 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-edd0a5a2-4912-4489-a5de-a072a94f900e None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:09.377267 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-edd0a5a2-4912-4489-a5de-a072a94f900e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:09.411019 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1201/2386] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:09.412319 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420c55e1-0b3b-4fb1-85fc-bc0f9a51aa54 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:09.423777 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420c55e1-0b3b-4fb1-85fc-bc0f9a51aa54 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:09.428681 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2e11a4-6418-4c50-bc5e-5c1d885f8d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:09.429289 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e11a4-6418-4c50-bc5e-5c1d885f8d1f admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:09.429560 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e11a4-6418-4c50-bc5e-5c1d885f8d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:09.429823 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2e11a4-6418-4c50-bc5e-5c1d885f8d1f admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:09.430498 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2e11a4-6418-4c50-bc5e-5c1d885f8d1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:48:09.430800 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2e11a4-6418-4c50-bc5e-5c1d885f8d1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:09.432390 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2e11a4-6418-4c50-bc5e-5c1d885f8d1f admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:09.445659 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1186/2387] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:48:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:48:09.450068 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd0a5a2-4912-4489-a5de-a072a94f900e None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:09.460422 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd0a5a2-4912-4489-a5de-a072a94f900e None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:09.465255 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f92fd7-2d6e-417f-9325-f6ef9df2836e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:09.465961 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f92fd7-2d6e-417f-9325-f6ef9df2836e admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:09.466456 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f92fd7-2d6e-417f-9325-f6ef9df2836e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:09.466787 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f92fd7-2d6e-417f-9325-f6ef9df2836e admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:09.467476 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f92fd7-2d6e-417f-9325-f6ef9df2836e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:09.469394 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f92fd7-2d6e-417f-9325-f6ef9df2836e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:09.497646 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1202/2388] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:48:09 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:48:09.502106 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2e11a4-6418-4c50-bc5e-5c1d885f8d1f admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:09.512815 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2e11a4-6418-4c50-bc5e-5c1d885f8d1f admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:09.579547 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8c50e9-aad4-4be0-beec-2c9b51b7341e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:09.580255 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c50e9-aad4-4be0-beec-2c9b51b7341e admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:09.580444 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c50e9-aad4-4be0-beec-2c9b51b7341e admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:09.580715 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c50e9-aad4-4be0-beec-2c9b51b7341e admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:09.581339 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8c50e9-aad4-4be0-beec-2c9b51b7341e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:09.583318 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8c50e9-aad4-4be0-beec-2c9b51b7341e admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:09.609139 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1187/2389] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:48:09 2022] POST /identity/v3/users => generated 287 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:48:09.613274 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f92fd7-2d6e-417f-9325-f6ef9df2836e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:09.624855 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f92fd7-2d6e-417f-9325-f6ef9df2836e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:09.629930 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa2f6f1-5e9d-48e0-8ba7-a0b702021246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:09.631111 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2f6f1-5e9d-48e0-8ba7-a0b702021246 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:09.631420 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2f6f1-5e9d-48e0-8ba7-a0b702021246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:09.631717 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2f6f1-5e9d-48e0-8ba7-a0b702021246 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:09.632360 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa2f6f1-5e9d-48e0-8ba7-a0b702021246 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:09.634020 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa2f6f1-5e9d-48e0-8ba7-a0b702021246 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:09.644924 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1203/2390] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:48:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:48:09.648771 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8c50e9-aad4-4be0-beec-2c9b51b7341e admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:09.659240 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8c50e9-aad4-4be0-beec-2c9b51b7341e admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:09.664295 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e83511b-ef27-407c-8a09-0ea62a5893fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:09.664910 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83511b-ef27-407c-8a09-0ea62a5893fe admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:09.665177 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83511b-ef27-407c-8a09-0ea62a5893fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:09.665438 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83511b-ef27-407c-8a09-0ea62a5893fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:09.666104 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e83511b-ef27-407c-8a09-0ea62a5893fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:09.667688 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e83511b-ef27-407c-8a09-0ea62a5893fe admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:09.678514 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1188/2391] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:48:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:48:09.682669 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa2f6f1-5e9d-48e0-8ba7-a0b702021246 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:09.692483 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa2f6f1-5e9d-48e0-8ba7-a0b702021246 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:09.697694 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5b1775-f517-43f1-9d13-fbcc00525a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:09.698381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b1775-f517-43f1-9d13-fbcc00525a43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:09.698663 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b1775-f517-43f1-9d13-fbcc00525a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:09.698932 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5b1775-f517-43f1-9d13-fbcc00525a43 admin admin] PATH_INFO: `/v3/projects/717df99b46ff4d30b2eb851313419101/users/9e99b41b05c24bad90fb41c0a1e770c4/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:09.715462 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5b1775-f517-43f1-9d13-fbcc00525a43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=717df99b46ff4d30b2eb851313419101, user_id=9e99b41b05c24bad90fb41c0a1e770c4, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:09.718178 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5b1775-f517-43f1-9d13-fbcc00525a43 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:09.732759 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1204/2392] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:48:09 2022] PUT /identity/v3/projects/717df99b46ff4d30b2eb851313419101/users/9e99b41b05c24bad90fb41c0a1e770c4/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:09.744696 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3451051a-d301-46bc-bb85-a8eb85f0ce95 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:09.744982 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3451051a-d301-46bc-bb85-a8eb85f0ce95 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:09.745253 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3451051a-d301-46bc-bb85-a8eb85f0ce95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:09.776887 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-3451051a-d301-46bc-bb85-a8eb85f0ce95 None None] MFA Rules not processed for user `9e99b41b05c24bad90fb41c0a1e770c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:09.778146 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3451051a-d301-46bc-bb85-a8eb85f0ce95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:09.848207 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1189/2393] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:09.852197 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5b1775-f517-43f1-9d13-fbcc00525a43 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:09.941577 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5b1775-f517-43f1-9d13-fbcc00525a43 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:09.947128 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12705ac0-1d6f-4d8f-b7bb-cbe74387950a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:09.947736 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12705ac0-1d6f-4d8f-b7bb-cbe74387950a admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:09.948026 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12705ac0-1d6f-4d8f-b7bb-cbe74387950a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:09.951477 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12705ac0-1d6f-4d8f-b7bb-cbe74387950a admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:09.951477 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12705ac0-1d6f-4d8f-b7bb-cbe74387950a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:09.951990 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12705ac0-1d6f-4d8f-b7bb-cbe74387950a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:10.001544 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1205/2394] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:48:09 2022] POST /identity/v3/projects => generated 371 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:48:10.005429 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3451051a-d301-46bc-bb85-a8eb85f0ce95 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:10.017040 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3451051a-d301-46bc-bb85-a8eb85f0ce95 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:10.050261 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:10.075228 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c968635f-5a35-4307-9609-2ea56438655a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:10.075839 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c968635f-5a35-4307-9609-2ea56438655a admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:10.076119 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c968635f-5a35-4307-9609-2ea56438655a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:10.076396 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c968635f-5a35-4307-9609-2ea56438655a admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:10.077132 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c968635f-5a35-4307-9609-2ea56438655a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:10.079465 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c968635f-5a35-4307-9609-2ea56438655a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:10.101441 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1190/2395] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:48:10 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:48:10.110819 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12705ac0-1d6f-4d8f-b7bb-cbe74387950a admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:10.121982 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12705ac0-1d6f-4d8f-b7bb-cbe74387950a admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:10.126591 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f68512-2248-4f1e-8a4a-efd6a83fda4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:10.127187 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f68512-2248-4f1e-8a4a-efd6a83fda4f admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:10.127477 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f68512-2248-4f1e-8a4a-efd6a83fda4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:10.127753 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f68512-2248-4f1e-8a4a-efd6a83fda4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:10.128389 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f68512-2248-4f1e-8a4a-efd6a83fda4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:10.130092 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f68512-2248-4f1e-8a4a-efd6a83fda4f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:10.144274 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1206/2396] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:48:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:48:10.145690 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c968635f-5a35-4307-9609-2ea56438655a admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:10.158516 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c968635f-5a35-4307-9609-2ea56438655a admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:10.163512 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f437a75-6d12-4692-8c56-2cd33a0120ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:10.164307 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f437a75-6d12-4692-8c56-2cd33a0120ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:10.164587 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f437a75-6d12-4692-8c56-2cd33a0120ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:10.164844 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f437a75-6d12-4692-8c56-2cd33a0120ac admin admin] PATH_INFO: `/v3/projects/c30822ad9ef84e219e3bced9fca407f9/users/ba8dfd4a51864e3c96911f3cbaa93913/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:10.181136 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f437a75-6d12-4692-8c56-2cd33a0120ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=c30822ad9ef84e219e3bced9fca407f9, user_id=ba8dfd4a51864e3c96911f3cbaa93913, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:10.183804 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f437a75-6d12-4692-8c56-2cd33a0120ac admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:10.197019 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1191/2397] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:48:10 2022] PUT /identity/v3/projects/c30822ad9ef84e219e3bced9fca407f9/users/ba8dfd4a51864e3c96911f3cbaa93913/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:10.205821 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f68512-2248-4f1e-8a4a-efd6a83fda4f admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:10.215695 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f68512-2248-4f1e-8a4a-efd6a83fda4f admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:10.272291 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422b2488-0598-452a-bba9-672f7e955ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:10.272901 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b2488-0598-452a-bba9-672f7e955ca6 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:10.273178 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b2488-0598-452a-bba9-672f7e955ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:10.273446 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b2488-0598-452a-bba9-672f7e955ca6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:10.274112 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422b2488-0598-452a-bba9-672f7e955ca6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:10.275702 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422b2488-0598-452a-bba9-672f7e955ca6 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:10.277904 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:10.291035 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1207/2398] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:48:10 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:48:10.292025 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f437a75-6d12-4692-8c56-2cd33a0120ac admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:10.303236 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f437a75-6d12-4692-8c56-2cd33a0120ac admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:10.307908 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a754a-13a7-4920-b967-68b562813702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:10.308549 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a754a-13a7-4920-b967-68b562813702 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:10.308819 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a754a-13a7-4920-b967-68b562813702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:10.309109 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a754a-13a7-4920-b967-68b562813702 admin admin] PATH_INFO: `/v3/projects/c30822ad9ef84e219e3bced9fca407f9/users/ba8dfd4a51864e3c96911f3cbaa93913/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:10.310989 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816a754a-13a7-4920-b967-68b562813702 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c30822ad9ef84e219e3bced9fca407f9, user_id=ba8dfd4a51864e3c96911f3cbaa93913, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:10.313569 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816a754a-13a7-4920-b967-68b562813702 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:10.323511 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1192/2399] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:48:10 2022] PUT /identity/v3/projects/c30822ad9ef84e219e3bced9fca407f9/users/ba8dfd4a51864e3c96911f3cbaa93913/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:48:10.334318 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9be1fd-7685-4a48-ad7f-28146bb6b445 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:10.334608 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9be1fd-7685-4a48-ad7f-28146bb6b445 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:10.334886 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9be1fd-7685-4a48-ad7f-28146bb6b445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:10.362756 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-aa9be1fd-7685-4a48-ad7f-28146bb6b445 None None] MFA Rules not processed for user `ba8dfd4a51864e3c96911f3cbaa93913`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:10.363969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-aa9be1fd-7685-4a48-ad7f-28146bb6b445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:10.430953 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1208/2400] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:10 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:10.442565 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a754a-13a7-4920-b967-68b562813702 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:10.444587 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-816a754a-13a7-4920-b967-68b562813702 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:10.503196 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a754a-13a7-4920-b967-68b562813702 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:10.506754 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47e40ee-bf36-49ce-b607-bf108914baf0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:10.507345 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e40ee-bf36-49ce-b607-bf108914baf0 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:10.507616 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e40ee-bf36-49ce-b607-bf108914baf0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:10.507878 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e40ee-bf36-49ce-b607-bf108914baf0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:10.516595 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47e40ee-bf36-49ce-b607-bf108914baf0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:10.518036 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47e40ee-bf36-49ce-b607-bf108914baf0 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:10.527519 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1193/2401] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:48:10 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:48:10.924755 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9be1fd-7685-4a48-ad7f-28146bb6b445 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:10.992140 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9be1fd-7685-4a48-ad7f-28146bb6b445 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:10.997153 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55cae38-ffb9-467e-8e0c-0501a83e87e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:10.997783 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55cae38-ffb9-467e-8e0c-0501a83e87e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:10.998184 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55cae38-ffb9-467e-8e0c-0501a83e87e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:10.998371 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55cae38-ffb9-467e-8e0c-0501a83e87e0 admin admin] PATH_INFO: `/v3/users/9e99b41b05c24bad90fb41c0a1e770c4` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:10.999581 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55cae38-ffb9-467e-8e0c-0501a83e87e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e99b41b05c24bad90fb41c0a1e770c4)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:11.001510 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55cae38-ffb9-467e-8e0c-0501a83e87e0 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:11.068499 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e55cae38-ffb9-467e-8e0c-0501a83e87e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e99b41b05c24bad90fb41c0a1e770c4'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:11.078463 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e55cae38-ffb9-467e-8e0c-0501a83e87e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e99b41b05c24bad90fb41c0a1e770c4'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:11.114076 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e55cae38-ffb9-467e-8e0c-0501a83e87e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e99b41b05c24bad90fb41c0a1e770c4'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:11.139523 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e55cae38-ffb9-467e-8e0c-0501a83e87e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e99b41b05c24bad90fb41c0a1e770c4'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:11.147918 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e55cae38-ffb9-467e-8e0c-0501a83e87e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e99b41b05c24bad90fb41c0a1e770c4'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:11.154926 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1209/2402] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:48:10 2022] DELETE /identity/v3/users/9e99b41b05c24bad90fb41c0a1e770c4 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:11.165790 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ded25-4f3b-43ba-9dbd-3e45b97a2239 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:11.166305 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ded25-4f3b-43ba-9dbd-3e45b97a2239 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:11.166461 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ded25-4f3b-43ba-9dbd-3e45b97a2239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:11.187161 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-167ded25-4f3b-43ba-9dbd-3e45b97a2239 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:11.188394 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-167ded25-4f3b-43ba-9dbd-3e45b97a2239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:11.251425 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1194/2403] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:11.261601 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55cae38-ffb9-467e-8e0c-0501a83e87e0 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:11.263114 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e55cae38-ffb9-467e-8e0c-0501a83e87e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:11.268366 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:11.323644 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55cae38-ffb9-467e-8e0c-0501a83e87e0 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:11.327222 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f72ac7-3970-409a-8672-08b624114b7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:11.327819 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f72ac7-3970-409a-8672-08b624114b7b service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:11.328097 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f72ac7-3970-409a-8672-08b624114b7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:11.328377 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f72ac7-3970-409a-8672-08b624114b7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:11.337279 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f72ac7-3970-409a-8672-08b624114b7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:11.338736 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f72ac7-3970-409a-8672-08b624114b7b service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:11.347970 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1210/2404] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:48:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:48:11.624392 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167ded25-4f3b-43ba-9dbd-3e45b97a2239 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:11.625829 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-167ded25-4f3b-43ba-9dbd-3e45b97a2239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:11.643758 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167ded25-4f3b-43ba-9dbd-3e45b97a2239 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:11.648833 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f9a10-f3f2-4d9c-bd27-8ba4a390cd0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:11.649627 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f9a10-f3f2-4d9c-bd27-8ba4a390cd0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:11.649932 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f9a10-f3f2-4d9c-bd27-8ba4a390cd0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:11.650200 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f9a10-f3f2-4d9c-bd27-8ba4a390cd0f admin admin] PATH_INFO: `/v3/projects/717df99b46ff4d30b2eb851313419101` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:11.651317 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939f9a10-f3f2-4d9c-bd27-8ba4a390cd0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=717df99b46ff4d30b2eb851313419101)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:11.653190 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939f9a10-f3f2-4d9c-bd27-8ba4a390cd0f admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:11.684802 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:11.702129 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-939f9a10-f3f2-4d9c-bd27-8ba4a390cd0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '717df99b46ff4d30b2eb851313419101', 'request_id': 'req-939f9a10-f3f2-4d9c-bd27-8ba4a390cd0f'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:11.709502 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-939f9a10-f3f2-4d9c-bd27-8ba4a390cd0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '717df99b46ff4d30b2eb851313419101', 'request_id': 'req-939f9a10-f3f2-4d9c-bd27-8ba4a390cd0f'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:11.715855 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-939f9a10-f3f2-4d9c-bd27-8ba4a390cd0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '717df99b46ff4d30b2eb851313419101', 'request_id': 'req-939f9a10-f3f2-4d9c-bd27-8ba4a390cd0f'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:11.726617 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-939f9a10-f3f2-4d9c-bd27-8ba4a390cd0f admin admin] The token cache is being invalidate because project 717df99b46ff4d30b2eb851313419101 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:48:11.726900 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-939f9a10-f3f2-4d9c-bd27-8ba4a390cd0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:11.727548 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-939f9a10-f3f2-4d9c-bd27-8ba4a390cd0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:11.728999 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1195/2405] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:48:11 2022] DELETE /identity/v3/projects/717df99b46ff4d30b2eb851313419101 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:11.733016 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f72ac7-3970-409a-8672-08b624114b7b service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:11.743393 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f72ac7-3970-409a-8672-08b624114b7b service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:11.797319 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aac04b-fe6d-4a24-95f0-286b13aae2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:11.797974 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aac04b-fe6d-4a24-95f0-286b13aae2af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:11.798256 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aac04b-fe6d-4a24-95f0-286b13aae2af admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:11.798527 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aac04b-fe6d-4a24-95f0-286b13aae2af admin admin] PATH_INFO: `/v3/users/ba8dfd4a51864e3c96911f3cbaa93913` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:11.799620 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aac04b-fe6d-4a24-95f0-286b13aae2af admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba8dfd4a51864e3c96911f3cbaa93913)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:11.801534 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aac04b-fe6d-4a24-95f0-286b13aae2af admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:11.845440 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-83aac04b-fe6d-4a24-95f0-286b13aae2af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba8dfd4a51864e3c96911f3cbaa93913'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:11.846114 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-83aac04b-fe6d-4a24-95f0-286b13aae2af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba8dfd4a51864e3c96911f3cbaa93913'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:11.858474 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-83aac04b-fe6d-4a24-95f0-286b13aae2af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba8dfd4a51864e3c96911f3cbaa93913'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:11.865819 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-83aac04b-fe6d-4a24-95f0-286b13aae2af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba8dfd4a51864e3c96911f3cbaa93913'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:11.872683 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-83aac04b-fe6d-4a24-95f0-286b13aae2af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba8dfd4a51864e3c96911f3cbaa93913'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:11.878406 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1211/2406] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:48:11 2022] DELETE /identity/v3/users/ba8dfd4a51864e3c96911f3cbaa93913 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:12.141601 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f9a10-f3f2-4d9c-bd27-8ba4a390cd0f admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:12.143790 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-939f9a10-f3f2-4d9c-bd27-8ba4a390cd0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:12.160239 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f9a10-f3f2-4d9c-bd27-8ba4a390cd0f admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:12.223229 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57afc4d7-8047-4306-8609-9cf53c890136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:12.224459 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57afc4d7-8047-4306-8609-9cf53c890136 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:12.224642 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57afc4d7-8047-4306-8609-9cf53c890136 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:12.224901 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57afc4d7-8047-4306-8609-9cf53c890136 admin admin] PATH_INFO: `/v3/projects/c30822ad9ef84e219e3bced9fca407f9` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:12.226159 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57afc4d7-8047-4306-8609-9cf53c890136 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c30822ad9ef84e219e3bced9fca407f9)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:12.227980 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57afc4d7-8047-4306-8609-9cf53c890136 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:12.284927 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-57afc4d7-8047-4306-8609-9cf53c890136 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c30822ad9ef84e219e3bced9fca407f9', 'request_id': 'req-57afc4d7-8047-4306-8609-9cf53c890136'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:12.302715 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-57afc4d7-8047-4306-8609-9cf53c890136 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c30822ad9ef84e219e3bced9fca407f9', 'request_id': 'req-57afc4d7-8047-4306-8609-9cf53c890136'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:12.309642 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-57afc4d7-8047-4306-8609-9cf53c890136 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c30822ad9ef84e219e3bced9fca407f9', 'request_id': 'req-57afc4d7-8047-4306-8609-9cf53c890136'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:12.321482 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-57afc4d7-8047-4306-8609-9cf53c890136 admin admin] The token cache is being invalidate because project c30822ad9ef84e219e3bced9fca407f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:48:12.321770 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-57afc4d7-8047-4306-8609-9cf53c890136 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:12.322504 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-57afc4d7-8047-4306-8609-9cf53c890136 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:12.323971 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1196/2407] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:48:12 2022] DELETE /identity/v3/projects/c30822ad9ef84e219e3bced9fca407f9 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:12.328818 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6f2e4-2984-4bce-bb96-26d911581ac8 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:12.329309 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6f2e4-2984-4bce-bb96-26d911581ac8 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:12.329418 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6f2e4-2984-4bce-bb96-26d911581ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:12.350008 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-38a6f2e4-2984-4bce-bb96-26d911581ac8 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:12.351254 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-38a6f2e4-2984-4bce-bb96-26d911581ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:12.370518 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:12.419867 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1212/2408] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:12.428445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef4e2c9-b781-47db-9c02-18b7c3238e59 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:12.428751 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef4e2c9-b781-47db-9c02-18b7c3238e59 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:12.429039 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef4e2c9-b781-47db-9c02-18b7c3238e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:12.449541 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-0ef4e2c9-b781-47db-9c02-18b7c3238e59 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:12.450779 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-0ef4e2c9-b781-47db-9c02-18b7c3238e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:12.461106 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1197/2409] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:12.465574 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a6f2e4-2984-4bce-bb96-26d911581ac8 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:12.475715 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a6f2e4-2984-4bce-bb96-26d911581ac8 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:12.480537 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a1f027-f798-4378-859c-e4226a89940f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:12.481163 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a1f027-f798-4378-859c-e4226a89940f admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:12.481439 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a1f027-f798-4378-859c-e4226a89940f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:12.481708 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a1f027-f798-4378-859c-e4226a89940f admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:12.482377 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a1f027-f798-4378-859c-e4226a89940f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:48:12.482678 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a1f027-f798-4378-859c-e4226a89940f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:12.484288 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a1f027-f798-4378-859c-e4226a89940f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:12.495868 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1213/2410] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:48:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:48:12.500124 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef4e2c9-b781-47db-9c02-18b7c3238e59 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:12.509871 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef4e2c9-b781-47db-9c02-18b7c3238e59 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:12.514866 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78c745c-3163-4872-93a8-0111fc8cf779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:12.515448 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c745c-3163-4872-93a8-0111fc8cf779 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:12.515722 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c745c-3163-4872-93a8-0111fc8cf779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:12.515988 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78c745c-3163-4872-93a8-0111fc8cf779 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:12.516615 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78c745c-3163-4872-93a8-0111fc8cf779 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:12.518537 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78c745c-3163-4872-93a8-0111fc8cf779 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:12.545759 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1198/2411] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:48:12 2022] POST /identity/v3/projects => generated 361 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:48:12.549839 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a1f027-f798-4378-859c-e4226a89940f admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:12.559711 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a1f027-f798-4378-859c-e4226a89940f admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:12.614135 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ab685d-0218-434a-85e2-3879cc2dd50b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:12.615563 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab685d-0218-434a-85e2-3879cc2dd50b admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:12.615837 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab685d-0218-434a-85e2-3879cc2dd50b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:12.616307 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab685d-0218-434a-85e2-3879cc2dd50b admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:12.616924 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ab685d-0218-434a-85e2-3879cc2dd50b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:12.618856 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ab685d-0218-434a-85e2-3879cc2dd50b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:12.640150 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1214/2412] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:48:12 2022] POST /identity/v3/users => generated 281 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:48:12.644240 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78c745c-3163-4872-93a8-0111fc8cf779 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:12.654235 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78c745c-3163-4872-93a8-0111fc8cf779 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:12.658751 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c086c5c-30ff-4bc4-9fb8-93c618b5ea10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:12.659333 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c086c5c-30ff-4bc4-9fb8-93c618b5ea10 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:12.659608 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c086c5c-30ff-4bc4-9fb8-93c618b5ea10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:12.659876 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c086c5c-30ff-4bc4-9fb8-93c618b5ea10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:12.660501 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c086c5c-30ff-4bc4-9fb8-93c618b5ea10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:12.662132 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c086c5c-30ff-4bc4-9fb8-93c618b5ea10 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:12.671791 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1199/2413] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:48:12 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:48:12.675518 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ab685d-0218-434a-85e2-3879cc2dd50b admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:12.685829 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ab685d-0218-434a-85e2-3879cc2dd50b admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:12.690541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380e8128-19b9-4885-9613-118cbdb33a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:12.691148 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e8128-19b9-4885-9613-118cbdb33a6b admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:12.691449 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e8128-19b9-4885-9613-118cbdb33a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:12.691742 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380e8128-19b9-4885-9613-118cbdb33a6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:12.692363 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380e8128-19b9-4885-9613-118cbdb33a6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:12.694614 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380e8128-19b9-4885-9613-118cbdb33a6b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:12.705220 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1215/2414] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:48:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:48:12.709321 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c086c5c-30ff-4bc4-9fb8-93c618b5ea10 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:12.719682 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c086c5c-30ff-4bc4-9fb8-93c618b5ea10 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:12.724278 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cbc44b-aca3-41bb-a664-0d37b00df672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:12.724983 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cbc44b-aca3-41bb-a664-0d37b00df672 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:12.725318 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cbc44b-aca3-41bb-a664-0d37b00df672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:12.725648 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cbc44b-aca3-41bb-a664-0d37b00df672 admin admin] PATH_INFO: `/v3/projects/56a1338ab7014ea3b7b1bbc4b6326487/users/51d6885541774412b0895468d25de332/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:12.727973 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:12.744352 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cbc44b-aca3-41bb-a664-0d37b00df672 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56a1338ab7014ea3b7b1bbc4b6326487, user_id=51d6885541774412b0895468d25de332, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:12.747014 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cbc44b-aca3-41bb-a664-0d37b00df672 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:12.767300 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1200/2415] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:48:12 2022] PUT /identity/v3/projects/56a1338ab7014ea3b7b1bbc4b6326487/users/51d6885541774412b0895468d25de332/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:48:12.770567 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036600d-d8d0-4157-b6e5-0608d0353087 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:12.770850 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036600d-d8d0-4157-b6e5-0608d0353087 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:12.771118 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036600d-d8d0-4157-b6e5-0608d0353087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:12.798013 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d036600d-d8d0-4157-b6e5-0608d0353087 None None] MFA Rules not processed for user `51d6885541774412b0895468d25de332`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:12.799221 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d036600d-d8d0-4157-b6e5-0608d0353087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:12.867705 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1216/2416] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:12.872176 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cbc44b-aca3-41bb-a664-0d37b00df672 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:12.940509 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cbc44b-aca3-41bb-a664-0d37b00df672 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:12.946138 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33d2781-b223-4235-beb9-d9b13040a85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:12.946775 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d2781-b223-4235-beb9-d9b13040a85b admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:12.947048 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d2781-b223-4235-beb9-d9b13040a85b admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:12.947316 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d2781-b223-4235-beb9-d9b13040a85b admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:12.948604 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33d2781-b223-4235-beb9-d9b13040a85b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:12.950756 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33d2781-b223-4235-beb9-d9b13040a85b admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:12.977644 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1201/2417] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:48:12 2022] POST /identity/v3/projects => generated 365 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:48:12.981753 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d036600d-d8d0-4157-b6e5-0608d0353087 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:12.992849 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d036600d-d8d0-4157-b6e5-0608d0353087 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:13.008687 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:13.049332 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd7f3a1-0e26-4449-9687-dc8b5076a917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:13.049972 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7f3a1-0e26-4449-9687-dc8b5076a917 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:13.050254 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7f3a1-0e26-4449-9687-dc8b5076a917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:13.050523 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7f3a1-0e26-4449-9687-dc8b5076a917 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:13.051150 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd7f3a1-0e26-4449-9687-dc8b5076a917 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:13.053643 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd7f3a1-0e26-4449-9687-dc8b5076a917 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:13.075257 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1217/2418] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:48:12 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:48:13.079450 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33d2781-b223-4235-beb9-d9b13040a85b admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:13.089991 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33d2781-b223-4235-beb9-d9b13040a85b admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:13.094575 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1175c04c-8c87-4777-8c08-02426aa85037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:13.095164 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1175c04c-8c87-4777-8c08-02426aa85037 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:13.095436 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1175c04c-8c87-4777-8c08-02426aa85037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:13.095701 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1175c04c-8c87-4777-8c08-02426aa85037 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:13.096334 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1175c04c-8c87-4777-8c08-02426aa85037 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:13.097945 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1175c04c-8c87-4777-8c08-02426aa85037 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:13.109114 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1202/2419] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:48:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:48:13.113179 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd7f3a1-0e26-4449-9687-dc8b5076a917 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:13.123915 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd7f3a1-0e26-4449-9687-dc8b5076a917 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:13.128494 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825219b3-0c2b-41a9-8b14-bdd0b86a64a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:13.129141 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825219b3-0c2b-41a9-8b14-bdd0b86a64a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:13.129418 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825219b3-0c2b-41a9-8b14-bdd0b86a64a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:13.129687 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825219b3-0c2b-41a9-8b14-bdd0b86a64a0 admin admin] PATH_INFO: `/v3/projects/158a5e0903ee4aecb59014dbfcb3f1d9/users/a32b8ba1738343e49a5417e7c53b0f1c/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:13.145372 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825219b3-0c2b-41a9-8b14-bdd0b86a64a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=158a5e0903ee4aecb59014dbfcb3f1d9, user_id=a32b8ba1738343e49a5417e7c53b0f1c, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:13.146837 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825219b3-0c2b-41a9-8b14-bdd0b86a64a0 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:13.160851 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1218/2420] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:48:13 2022] PUT /identity/v3/projects/158a5e0903ee4aecb59014dbfcb3f1d9/users/a32b8ba1738343e49a5417e7c53b0f1c/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:48:13.166999 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1175c04c-8c87-4777-8c08-02426aa85037 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:13.179110 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1175c04c-8c87-4777-8c08-02426aa85037 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:13.250256 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd582929-237f-426c-97df-657794a7d6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:13.250943 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd582929-237f-426c-97df-657794a7d6ce admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:13.251219 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd582929-237f-426c-97df-657794a7d6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:13.251492 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd582929-237f-426c-97df-657794a7d6ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:13.252156 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd582929-237f-426c-97df-657794a7d6ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:13.253733 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd582929-237f-426c-97df-657794a7d6ce admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:13.264573 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1203/2421] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:48:13 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:48:13.268677 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825219b3-0c2b-41a9-8b14-bdd0b86a64a0 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:13.279081 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825219b3-0c2b-41a9-8b14-bdd0b86a64a0 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:13.284031 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3859062-4447-4e06-ae03-409e295d269f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:13.284683 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3859062-4447-4e06-ae03-409e295d269f admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:13.284971 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3859062-4447-4e06-ae03-409e295d269f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:13.285242 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3859062-4447-4e06-ae03-409e295d269f admin admin] PATH_INFO: `/v3/projects/158a5e0903ee4aecb59014dbfcb3f1d9/users/a32b8ba1738343e49a5417e7c53b0f1c/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:13.287831 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3859062-4447-4e06-ae03-409e295d269f admin admin] RBAC: Authorizing `identity:create_grant(project_id=158a5e0903ee4aecb59014dbfcb3f1d9, user_id=a32b8ba1738343e49a5417e7c53b0f1c, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:13.290547 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3859062-4447-4e06-ae03-409e295d269f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:13.301075 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1219/2422] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:48:13 2022] PUT /identity/v3/projects/158a5e0903ee4aecb59014dbfcb3f1d9/users/a32b8ba1738343e49a5417e7c53b0f1c/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:48:13.311772 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb193c4-efc8-40a4-a590-d1a220992905 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:13.312063 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb193c4-efc8-40a4-a590-d1a220992905 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:13.312361 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb193c4-efc8-40a4-a590-d1a220992905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:13.332487 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3859062-4447-4e06-ae03-409e295d269f admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:13.335284 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c3859062-4447-4e06-ae03-409e295d269f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:13.341614 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-dfb193c4-efc8-40a4-a590-d1a220992905 None None] MFA Rules not processed for user `a32b8ba1738343e49a5417e7c53b0f1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:13.342902 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-dfb193c4-efc8-40a4-a590-d1a220992905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:13.386919 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:13.403006 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3859062-4447-4e06-ae03-409e295d269f admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:13.407763 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925ecd1b-c025-4d15-b2de-e3d21749991f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:13.408392 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ecd1b-c025-4d15-b2de-e3d21749991f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:13.408671 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ecd1b-c025-4d15-b2de-e3d21749991f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:13.408943 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ecd1b-c025-4d15-b2de-e3d21749991f admin admin] PATH_INFO: `/v3/users/a0803fd2f9934228b11295cdd8d443dd` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:13.410115 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925ecd1b-c025-4d15-b2de-e3d21749991f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0803fd2f9934228b11295cdd8d443dd)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:13.412752 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925ecd1b-c025-4d15-b2de-e3d21749991f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:13.415231 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1204/2423] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:13 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:13.426039 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb193c4-efc8-40a4-a590-d1a220992905 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:13.427443 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-dfb193c4-efc8-40a4-a590-d1a220992905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:13.459410 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-925ecd1b-c025-4d15-b2de-e3d21749991f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0803fd2f9934228b11295cdd8d443dd'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:13.460087 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-925ecd1b-c025-4d15-b2de-e3d21749991f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0803fd2f9934228b11295cdd8d443dd'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:13.474483 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-925ecd1b-c025-4d15-b2de-e3d21749991f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0803fd2f9934228b11295cdd8d443dd'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:13.483017 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-925ecd1b-c025-4d15-b2de-e3d21749991f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0803fd2f9934228b11295cdd8d443dd'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:13.484022 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb193c4-efc8-40a4-a590-d1a220992905 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:13.486260 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:13.488019 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e7ed8d-510d-4ac1-a3dc-858db8d4ca1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:13.488634 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e7ed8d-510d-4ac1-a3dc-858db8d4ca1a service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:13.488944 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e7ed8d-510d-4ac1-a3dc-858db8d4ca1a service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:13.489219 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e7ed8d-510d-4ac1-a3dc-858db8d4ca1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:13.495413 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-925ecd1b-c025-4d15-b2de-e3d21749991f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0803fd2f9934228b11295cdd8d443dd'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:13.499637 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e7ed8d-510d-4ac1-a3dc-858db8d4ca1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:13.501062 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e7ed8d-510d-4ac1-a3dc-858db8d4ca1a service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:13.513466 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1205/2425] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:48:13 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:48:13.513768 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1220/2425] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:48:13 2022] DELETE /identity/v3/users/a0803fd2f9934228b11295cdd8d443dd => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:48:13.514517 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a5706-c68c-44f0-9f0a-c98aecf9fca7 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:13.514794 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a5706-c68c-44f0-9f0a-c98aecf9fca7 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:13.515057 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a5706-c68c-44f0-9f0a-c98aecf9fca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:13.535912 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-a77a5706-c68c-44f0-9f0a-c98aecf9fca7 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:13.537802 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a77a5706-c68c-44f0-9f0a-c98aecf9fca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:13.612167 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925ecd1b-c025-4d15-b2de-e3d21749991f admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:13.612881 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1206/2426] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:13.613693 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-925ecd1b-c025-4d15-b2de-e3d21749991f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:13.622781 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77a5706-c68c-44f0-9f0a-c98aecf9fca7 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:13.624202 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a77a5706-c68c-44f0-9f0a-c98aecf9fca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:13.679343 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925ecd1b-c025-4d15-b2de-e3d21749991f admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:13.683523 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1998bab6-317c-4f92-b629-46e9380ff304 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:13.684129 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1998bab6-317c-4f92-b629-46e9380ff304 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:13.684409 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1998bab6-317c-4f92-b629-46e9380ff304 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:13.684682 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1998bab6-317c-4f92-b629-46e9380ff304 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:13.686505 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1998bab6-317c-4f92-b629-46e9380ff304 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:13.693609 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77a5706-c68c-44f0-9f0a-c98aecf9fca7 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:13.697443 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b54b227-4ae9-4fba-a6c2-730ed5994c84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:13.698074 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54b227-4ae9-4fba-a6c2-730ed5994c84 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:13.698352 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54b227-4ae9-4fba-a6c2-730ed5994c84 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:13.698621 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54b227-4ae9-4fba-a6c2-730ed5994c84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:13.707778 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b54b227-4ae9-4fba-a6c2-730ed5994c84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:13.709713 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b54b227-4ae9-4fba-a6c2-730ed5994c84 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:13.719129 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1207/2427] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:48:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:48:13.746964 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1998bab6-317c-4f92-b629-46e9380ff304 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:13.748486 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1998bab6-317c-4f92-b629-46e9380ff304 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:13.757939 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1221/2428] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:48:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:48:14.022890 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b54b227-4ae9-4fba-a6c2-730ed5994c84 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:14.024426 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-1b54b227-4ae9-4fba-a6c2-730ed5994c84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:14.037567 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b54b227-4ae9-4fba-a6c2-730ed5994c84 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:14.048474 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b101eae-45cc-4ddb-a482-fb897b075333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:14.049141 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b101eae-45cc-4ddb-a482-fb897b075333 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:14.049483 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b101eae-45cc-4ddb-a482-fb897b075333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:14.049626 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b101eae-45cc-4ddb-a482-fb897b075333 admin admin] PATH_INFO: `/v3/projects/ca6444aebef74b2fb52ee7b66e037393` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:14.050762 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b101eae-45cc-4ddb-a482-fb897b075333 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca6444aebef74b2fb52ee7b66e037393)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:14.052689 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b101eae-45cc-4ddb-a482-fb897b075333 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:14.074607 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:14.103669 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5b101eae-45cc-4ddb-a482-fb897b075333 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca6444aebef74b2fb52ee7b66e037393', 'request_id': 'req-5b101eae-45cc-4ddb-a482-fb897b075333'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:14.115521 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5b101eae-45cc-4ddb-a482-fb897b075333 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca6444aebef74b2fb52ee7b66e037393', 'request_id': 'req-5b101eae-45cc-4ddb-a482-fb897b075333'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:14.122793 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5b101eae-45cc-4ddb-a482-fb897b075333 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca6444aebef74b2fb52ee7b66e037393', 'request_id': 'req-5b101eae-45cc-4ddb-a482-fb897b075333'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:14.133904 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5b101eae-45cc-4ddb-a482-fb897b075333 admin admin] The token cache is being invalidate because project ca6444aebef74b2fb52ee7b66e037393 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:48:14.134189 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5b101eae-45cc-4ddb-a482-fb897b075333 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:14.134811 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-5b101eae-45cc-4ddb-a482-fb897b075333 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:14.136270 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1208/2429] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:48:14 2022] DELETE /identity/v3/projects/ca6444aebef74b2fb52ee7b66e037393 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:14.141122 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e00f5d5-7107-4f63-af96-ff46bdf9491d None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:14.141401 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e00f5d5-7107-4f63-af96-ff46bdf9491d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:14.141668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e00f5d5-7107-4f63-af96-ff46bdf9491d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:14.161744 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-4e00f5d5-7107-4f63-af96-ff46bdf9491d None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:14.162990 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4e00f5d5-7107-4f63-af96-ff46bdf9491d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:14.220334 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1222/2430] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:48:14.227972 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808695c-fab1-4594-93f3-1101ec5e2e5b None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:14.228259 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808695c-fab1-4594-93f3-1101ec5e2e5b None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:14.228553 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808695c-fab1-4594-93f3-1101ec5e2e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:14.249242 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-0808695c-fab1-4594-93f3-1101ec5e2e5b None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:14.250453 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-0808695c-fab1-4594-93f3-1101ec5e2e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:14.258813 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1209/2431] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:14.263008 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e00f5d5-7107-4f63-af96-ff46bdf9491d None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:14.274999 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e00f5d5-7107-4f63-af96-ff46bdf9491d None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:14.279876 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08a87ac-2720-4499-8115-fed0e82160e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:14.280480 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a87ac-2720-4499-8115-fed0e82160e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:14.280757 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a87ac-2720-4499-8115-fed0e82160e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:14.281027 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a87ac-2720-4499-8115-fed0e82160e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:14.281687 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08a87ac-2720-4499-8115-fed0e82160e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:48:14.282021 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08a87ac-2720-4499-8115-fed0e82160e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:14.283595 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08a87ac-2720-4499-8115-fed0e82160e9 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:14.295114 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1223/2432] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:48:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:48:14.299444 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0808695c-fab1-4594-93f3-1101ec5e2e5b None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:14.308924 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0808695c-fab1-4594-93f3-1101ec5e2e5b None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:14.313448 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95a39d6-5280-4eb5-847e-0aed8b56a679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:14.314063 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a39d6-5280-4eb5-847e-0aed8b56a679 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:14.314333 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a39d6-5280-4eb5-847e-0aed8b56a679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:14.314596 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a39d6-5280-4eb5-847e-0aed8b56a679 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:14.315205 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95a39d6-5280-4eb5-847e-0aed8b56a679 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:14.317067 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95a39d6-5280-4eb5-847e-0aed8b56a679 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:14.343000 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1210/2433] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:48:14 2022] POST /identity/v3/projects => generated 397 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:48:14.347037 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08a87ac-2720-4499-8115-fed0e82160e9 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:14.356973 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08a87ac-2720-4499-8115-fed0e82160e9 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:14.386550 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:14.413083 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8dd18e-2fb2-4aca-a756-4e29a34a9550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:14.413689 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8dd18e-2fb2-4aca-a756-4e29a34a9550 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:14.414001 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8dd18e-2fb2-4aca-a756-4e29a34a9550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:14.414271 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8dd18e-2fb2-4aca-a756-4e29a34a9550 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:14.414892 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8dd18e-2fb2-4aca-a756-4e29a34a9550 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:14.416789 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8dd18e-2fb2-4aca-a756-4e29a34a9550 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:14.441108 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1224/2434] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:48:14 2022] POST /identity/v3/users => generated 299 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:48:14.445949 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95a39d6-5280-4eb5-847e-0aed8b56a679 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:14.459897 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95a39d6-5280-4eb5-847e-0aed8b56a679 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:14.467251 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09fab6e-49c1-4948-839e-17b7b2a76399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:14.467854 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09fab6e-49c1-4948-839e-17b7b2a76399 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:14.468129 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09fab6e-49c1-4948-839e-17b7b2a76399 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:14.468391 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09fab6e-49c1-4948-839e-17b7b2a76399 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:14.469220 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09fab6e-49c1-4948-839e-17b7b2a76399 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:14.470822 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09fab6e-49c1-4948-839e-17b7b2a76399 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:14.483437 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1211/2435] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:48:14 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:48:14.487467 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8dd18e-2fb2-4aca-a756-4e29a34a9550 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:14.498546 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8dd18e-2fb2-4aca-a756-4e29a34a9550 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:14.503777 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad1ecd8-b79c-496b-a405-bb07abe462f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:14.504637 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad1ecd8-b79c-496b-a405-bb07abe462f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:14.504942 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad1ecd8-b79c-496b-a405-bb07abe462f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:14.505208 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad1ecd8-b79c-496b-a405-bb07abe462f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:14.505830 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad1ecd8-b79c-496b-a405-bb07abe462f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:14.508186 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad1ecd8-b79c-496b-a405-bb07abe462f9 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:14.519413 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1225/2436] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:48:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:48:14.523843 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09fab6e-49c1-4948-839e-17b7b2a76399 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:14.533654 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09fab6e-49c1-4948-839e-17b7b2a76399 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:14.538623 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d7db63-30b6-40a1-8a09-0c90c112a942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:14.539254 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d7db63-30b6-40a1-8a09-0c90c112a942 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:14.539528 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d7db63-30b6-40a1-8a09-0c90c112a942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:14.539797 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d7db63-30b6-40a1-8a09-0c90c112a942 admin admin] PATH_INFO: `/v3/projects/4dc4eb2d576c47d39e3f0fdb80b2c611/users/a95e88a42d784991b7e9926e764bc429/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:14.554685 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d7db63-30b6-40a1-8a09-0c90c112a942 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dc4eb2d576c47d39e3f0fdb80b2c611, user_id=a95e88a42d784991b7e9926e764bc429, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:14.557288 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d7db63-30b6-40a1-8a09-0c90c112a942 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:14.570320 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1212/2437] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:48:14 2022] PUT /identity/v3/projects/4dc4eb2d576c47d39e3f0fdb80b2c611/users/a95e88a42d784991b7e9926e764bc429/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:14.581230 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a143b-69bf-4098-a810-42689519ae49 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:14.581569 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a143b-69bf-4098-a810-42689519ae49 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:14.583862 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a143b-69bf-4098-a810-42689519ae49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:14.639778 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-b75a143b-69bf-4098-a810-42689519ae49 None None] MFA Rules not processed for user `a95e88a42d784991b7e9926e764bc429`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:14.642072 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b75a143b-69bf-4098-a810-42689519ae49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:14.748368 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1226/2438] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:14 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:14.761497 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d7db63-30b6-40a1-8a09-0c90c112a942 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:14.763242 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a0d7db63-30b6-40a1-8a09-0c90c112a942 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:14.847522 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d7db63-30b6-40a1-8a09-0c90c112a942 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:14.858527 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861c696d-febd-4466-9bda-45f17d07c626 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:14.858935 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c696d-febd-4466-9bda-45f17d07c626 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:14.859347 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c696d-febd-4466-9bda-45f17d07c626 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:14.859719 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c696d-febd-4466-9bda-45f17d07c626 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:14.872904 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861c696d-febd-4466-9bda-45f17d07c626 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:14.874807 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861c696d-febd-4466-9bda-45f17d07c626 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:14.887190 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1213/2439] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:48:14 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:48:16.251407 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c33404-8cc2-4c07-8062-08a300ea1495 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:16.251907 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c33404-8cc2-4c07-8062-08a300ea1495 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:16.252035 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c33404-8cc2-4c07-8062-08a300ea1495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:16.270701 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-f2c33404-8cc2-4c07-8062-08a300ea1495 None None] MFA Rules not processed for user `c81e5fb007614cf2992f5e02e85d6307`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:16.271888 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f2c33404-8cc2-4c07-8062-08a300ea1495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:16.326536 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1227/2440] 10.222.0.52 () {58 vars in 967 bytes} [Wed Sep 28 15:48:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:16.339136 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861c696d-febd-4466-9bda-45f17d07c626 service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:16.340494 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-861c696d-febd-4466-9bda-45f17d07c626 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:16.389778 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:16.413112 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861c696d-febd-4466-9bda-45f17d07c626 service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:16.417989 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb47854e-d857-4569-988d-c4fc4fa8fe6e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '328051fe7607464f8f5dc0dcc01a817a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:16.418581 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb47854e-d857-4569-988d-c4fc4fa8fe6e service swift] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:16.418851 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb47854e-d857-4569-988d-c4fc4fa8fe6e service swift] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:16.419144 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb47854e-d857-4569-988d-c4fc4fa8fe6e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:16.427749 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb47854e-d857-4569-988d-c4fc4fa8fe6e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:16.429346 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb47854e-d857-4569-988d-c4fc4fa8fe6e service swift] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:16.437509 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1214/2441] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 15:48:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 15:48:16.650788 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c33404-8cc2-4c07-8062-08a300ea1495 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:16.652545 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f2c33404-8cc2-4c07-8062-08a300ea1495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:16.733308 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:16.753928 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c33404-8cc2-4c07-8062-08a300ea1495 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:16.760742 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1180f4a-0b9f-444b-86b3-89e190ccf97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:16.761361 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1180f4a-0b9f-444b-86b3-89e190ccf97b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:16.761661 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1180f4a-0b9f-444b-86b3-89e190ccf97b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:16.761952 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1180f4a-0b9f-444b-86b3-89e190ccf97b admin admin] PATH_INFO: `/v3/users/51d6885541774412b0895468d25de332` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:16.763116 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1180f4a-0b9f-444b-86b3-89e190ccf97b admin admin] RBAC: Authorizing `identity:delete_user(user_id=51d6885541774412b0895468d25de332)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:16.765045 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1180f4a-0b9f-444b-86b3-89e190ccf97b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:16.814414 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d1180f4a-0b9f-444b-86b3-89e190ccf97b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51d6885541774412b0895468d25de332'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:16.815079 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d1180f4a-0b9f-444b-86b3-89e190ccf97b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51d6885541774412b0895468d25de332'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:16.827465 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d1180f4a-0b9f-444b-86b3-89e190ccf97b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51d6885541774412b0895468d25de332'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:16.836299 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d1180f4a-0b9f-444b-86b3-89e190ccf97b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51d6885541774412b0895468d25de332'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:16.843517 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-d1180f4a-0b9f-444b-86b3-89e190ccf97b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51d6885541774412b0895468d25de332'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:16.849527 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1228/2442] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:48:16 2022] DELETE /identity/v3/users/51d6885541774412b0895468d25de332 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:48:16.854039 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8eef20-56d4-4206-95f6-f6a6c615cbc5 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:16.854387 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8eef20-56d4-4206-95f6-f6a6c615cbc5 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:16.854726 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8eef20-56d4-4206-95f6-f6a6c615cbc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:16.873957 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-ef8eef20-56d4-4206-95f6-f6a6c615cbc5 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:16.875157 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ef8eef20-56d4-4206-95f6-f6a6c615cbc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:16.934154 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1215/2443] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:48:16.944288 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1180f4a-0b9f-444b-86b3-89e190ccf97b admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:16.945668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d1180f4a-0b9f-444b-86b3-89e190ccf97b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:17.000953 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1180f4a-0b9f-444b-86b3-89e190ccf97b admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:17.004485 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2efda0f-7030-4455-b418-221c110337aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:17.005614 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2efda0f-7030-4455-b418-221c110337aa service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:17.005925 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2efda0f-7030-4455-b418-221c110337aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:17.006201 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2efda0f-7030-4455-b418-221c110337aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:17.014952 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2efda0f-7030-4455-b418-221c110337aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:17.016361 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2efda0f-7030-4455-b418-221c110337aa service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:17.026141 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1229/2444] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:48:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:48:17.375059 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8eef20-56d4-4206-95f6-f6a6c615cbc5 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:17.376547 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ef8eef20-56d4-4206-95f6-f6a6c615cbc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:17.389255 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8eef20-56d4-4206-95f6-f6a6c615cbc5 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:17.393881 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417faace-0e19-4f70-a15c-41c5309f057a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:17.394509 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417faace-0e19-4f70-a15c-41c5309f057a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:17.394783 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417faace-0e19-4f70-a15c-41c5309f057a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:17.395061 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417faace-0e19-4f70-a15c-41c5309f057a admin admin] PATH_INFO: `/v3/projects/56a1338ab7014ea3b7b1bbc4b6326487` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:17.396166 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417faace-0e19-4f70-a15c-41c5309f057a admin admin] RBAC: Authorizing `identity:delete_project(project_id=56a1338ab7014ea3b7b1bbc4b6326487)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:17.398109 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417faace-0e19-4f70-a15c-41c5309f057a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:17.447567 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-417faace-0e19-4f70-a15c-41c5309f057a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56a1338ab7014ea3b7b1bbc4b6326487', 'request_id': 'req-417faace-0e19-4f70-a15c-41c5309f057a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:17.463196 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-417faace-0e19-4f70-a15c-41c5309f057a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56a1338ab7014ea3b7b1bbc4b6326487', 'request_id': 'req-417faace-0e19-4f70-a15c-41c5309f057a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:17.472041 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-417faace-0e19-4f70-a15c-41c5309f057a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56a1338ab7014ea3b7b1bbc4b6326487', 'request_id': 'req-417faace-0e19-4f70-a15c-41c5309f057a'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:17.483992 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-417faace-0e19-4f70-a15c-41c5309f057a admin admin] The token cache is being invalidate because project 56a1338ab7014ea3b7b1bbc4b6326487 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:48:17.484273 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-417faace-0e19-4f70-a15c-41c5309f057a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:17.484921 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-417faace-0e19-4f70-a15c-41c5309f057a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:17.486419 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1216/2445] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:48:17 2022] DELETE /identity/v3/projects/56a1338ab7014ea3b7b1bbc4b6326487 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:17.491042 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2efda0f-7030-4455-b418-221c110337aa service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:17.500530 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2efda0f-7030-4455-b418-221c110337aa service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:17.555403 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ffa3f-7d45-42dc-9759-60b3cd0f00e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:17.556023 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ffa3f-7d45-42dc-9759-60b3cd0f00e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:17.556302 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ffa3f-7d45-42dc-9759-60b3cd0f00e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:17.556570 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ffa3f-7d45-42dc-9759-60b3cd0f00e9 admin admin] PATH_INFO: `/v3/users/a32b8ba1738343e49a5417e7c53b0f1c` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:17.557655 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7ffa3f-7d45-42dc-9759-60b3cd0f00e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a32b8ba1738343e49a5417e7c53b0f1c)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:17.559645 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7ffa3f-7d45-42dc-9759-60b3cd0f00e9 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:17.569872 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:17.606736 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4d7ffa3f-7d45-42dc-9759-60b3cd0f00e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a32b8ba1738343e49a5417e7c53b0f1c'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:17.607375 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4d7ffa3f-7d45-42dc-9759-60b3cd0f00e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a32b8ba1738343e49a5417e7c53b0f1c'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:17.619583 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4d7ffa3f-7d45-42dc-9759-60b3cd0f00e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a32b8ba1738343e49a5417e7c53b0f1c'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:17.627043 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4d7ffa3f-7d45-42dc-9759-60b3cd0f00e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a32b8ba1738343e49a5417e7c53b0f1c'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:17.634970 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4d7ffa3f-7d45-42dc-9759-60b3cd0f00e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a32b8ba1738343e49a5417e7c53b0f1c'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:17.640892 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1230/2446] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:48:17 2022] DELETE /identity/v3/users/a32b8ba1738343e49a5417e7c53b0f1c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:17.913198 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417faace-0e19-4f70-a15c-41c5309f057a admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:17.914776 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-417faace-0e19-4f70-a15c-41c5309f057a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:17.927791 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417faace-0e19-4f70-a15c-41c5309f057a admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:17.983989 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42059d95-68ed-49d9-956b-c6612ed4a223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:17.984611 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42059d95-68ed-49d9-956b-c6612ed4a223 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:17.984887 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42059d95-68ed-49d9-956b-c6612ed4a223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:17.985158 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42059d95-68ed-49d9-956b-c6612ed4a223 admin admin] PATH_INFO: `/v3/projects/158a5e0903ee4aecb59014dbfcb3f1d9` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:17.986281 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42059d95-68ed-49d9-956b-c6612ed4a223 admin admin] RBAC: Authorizing `identity:delete_project(project_id=158a5e0903ee4aecb59014dbfcb3f1d9)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:17.988191 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42059d95-68ed-49d9-956b-c6612ed4a223 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:17.989874 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:18.035577 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-42059d95-68ed-49d9-956b-c6612ed4a223 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '158a5e0903ee4aecb59014dbfcb3f1d9', 'request_id': 'req-42059d95-68ed-49d9-956b-c6612ed4a223'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:18.043304 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-42059d95-68ed-49d9-956b-c6612ed4a223 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '158a5e0903ee4aecb59014dbfcb3f1d9', 'request_id': 'req-42059d95-68ed-49d9-956b-c6612ed4a223'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:18.049582 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-42059d95-68ed-49d9-956b-c6612ed4a223 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '158a5e0903ee4aecb59014dbfcb3f1d9', 'request_id': 'req-42059d95-68ed-49d9-956b-c6612ed4a223'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:18.059781 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-42059d95-68ed-49d9-956b-c6612ed4a223 admin admin] The token cache is being invalidate because project 158a5e0903ee4aecb59014dbfcb3f1d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:48:18.060006 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-42059d95-68ed-49d9-956b-c6612ed4a223 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:18.060624 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-42059d95-68ed-49d9-956b-c6612ed4a223 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:18.062124 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1217/2447] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:48:17 2022] DELETE /identity/v3/projects/158a5e0903ee4aecb59014dbfcb3f1d9 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:18.066941 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46374ce1-9ae2-4f2b-9b78-3da06d7bb646 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:18.067267 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46374ce1-9ae2-4f2b-9b78-3da06d7bb646 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:18.067545 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46374ce1-9ae2-4f2b-9b78-3da06d7bb646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:18.087722 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-46374ce1-9ae2-4f2b-9b78-3da06d7bb646 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:18.088922 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-46374ce1-9ae2-4f2b-9b78-3da06d7bb646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:18.147172 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1231/2448] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:18.156167 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d16d21d-cdda-40c1-833a-e8505bb65448 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:18.156451 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d16d21d-cdda-40c1-833a-e8505bb65448 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:18.156730 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d16d21d-cdda-40c1-833a-e8505bb65448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:18.176355 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-1d16d21d-cdda-40c1-833a-e8505bb65448 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:18.177547 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-1d16d21d-cdda-40c1-833a-e8505bb65448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:18.185261 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1218/2449] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:18.189582 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46374ce1-9ae2-4f2b-9b78-3da06d7bb646 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:18.199218 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46374ce1-9ae2-4f2b-9b78-3da06d7bb646 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:18.203784 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd4811f-1b7e-4918-97b3-7946b76e782d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:18.204374 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd4811f-1b7e-4918-97b3-7946b76e782d admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:18.204652 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd4811f-1b7e-4918-97b3-7946b76e782d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:18.204922 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd4811f-1b7e-4918-97b3-7946b76e782d admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:18.205548 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd4811f-1b7e-4918-97b3-7946b76e782d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:48:18.205867 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd4811f-1b7e-4918-97b3-7946b76e782d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:18.207450 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd4811f-1b7e-4918-97b3-7946b76e782d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:18.219432 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1232/2450] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:48:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:48:18.224491 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d16d21d-cdda-40c1-833a-e8505bb65448 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:18.233877 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d16d21d-cdda-40c1-833a-e8505bb65448 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:18.238876 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad38fd-f598-4795-ae11-0b284dcfcb38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:18.239723 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad38fd-f598-4795-ae11-0b284dcfcb38 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:18.239999 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad38fd-f598-4795-ae11-0b284dcfcb38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:18.240269 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad38fd-f598-4795-ae11-0b284dcfcb38 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:18.240899 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ad38fd-f598-4795-ae11-0b284dcfcb38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:18.242825 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ad38fd-f598-4795-ae11-0b284dcfcb38 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:18.274179 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1219/2451] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:48:18 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:48:18.275291 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd4811f-1b7e-4918-97b3-7946b76e782d admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:18.285400 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd4811f-1b7e-4918-97b3-7946b76e782d admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:18.342478 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550c71a-cbcf-409f-9d51-c06b05bd6cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:18.343003 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550c71a-cbcf-409f-9d51-c06b05bd6cd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:18.343285 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550c71a-cbcf-409f-9d51-c06b05bd6cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:18.343561 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550c71a-cbcf-409f-9d51-c06b05bd6cd5 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:18.344190 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0550c71a-cbcf-409f-9d51-c06b05bd6cd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:18.346146 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0550c71a-cbcf-409f-9d51-c06b05bd6cd5 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:18.367843 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1233/2452] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:48:18 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:48:18.371867 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad38fd-f598-4795-ae11-0b284dcfcb38 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:18.382567 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad38fd-f598-4795-ae11-0b284dcfcb38 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:18.387177 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff355f41-e3d8-4297-aa91-4cb87f234cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:18.387768 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff355f41-e3d8-4297-aa91-4cb87f234cb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:18.388044 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff355f41-e3d8-4297-aa91-4cb87f234cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:18.388315 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff355f41-e3d8-4297-aa91-4cb87f234cb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:18.388976 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff355f41-e3d8-4297-aa91-4cb87f234cb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:18.390620 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff355f41-e3d8-4297-aa91-4cb87f234cb0 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:18.401255 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1220/2453] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:48:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:48:18.405112 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550c71a-cbcf-409f-9d51-c06b05bd6cd5 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:18.411375 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:18.415583 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550c71a-cbcf-409f-9d51-c06b05bd6cd5 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:18.422111 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4d45f1-7b77-41ba-905a-72354d1e1a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:18.422905 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d45f1-7b77-41ba-905a-72354d1e1a27 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:18.423773 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d45f1-7b77-41ba-905a-72354d1e1a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:18.423773 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d45f1-7b77-41ba-905a-72354d1e1a27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:18.424441 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4d45f1-7b77-41ba-905a-72354d1e1a27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:18.426849 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4d45f1-7b77-41ba-905a-72354d1e1a27 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:18.438018 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1234/2454] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:48:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:48:18.442652 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff355f41-e3d8-4297-aa91-4cb87f234cb0 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:18.452547 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff355f41-e3d8-4297-aa91-4cb87f234cb0 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:18.457115 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2307e9b-3606-4a21-a85c-e10b4c83ef35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:18.457813 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2307e9b-3606-4a21-a85c-e10b4c83ef35 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:18.458643 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2307e9b-3606-4a21-a85c-e10b4c83ef35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:18.459344 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2307e9b-3606-4a21-a85c-e10b4c83ef35 admin admin] PATH_INFO: `/v3/projects/d2292df20aa847ff874c1afef13b917c/users/83f999f98b0b4cd6968bb24187a9f772/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:18.475387 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2307e9b-3606-4a21-a85c-e10b4c83ef35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2292df20aa847ff874c1afef13b917c, user_id=83f999f98b0b4cd6968bb24187a9f772, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:18.478337 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2307e9b-3606-4a21-a85c-e10b4c83ef35 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:18.497390 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1221/2455] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:48:18 2022] PUT /identity/v3/projects/d2292df20aa847ff874c1afef13b917c/users/83f999f98b0b4cd6968bb24187a9f772/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:18.508169 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a7aac-2a39-407d-ae9b-0eb63d88ff3d None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:18.508690 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a7aac-2a39-407d-ae9b-0eb63d88ff3d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:18.508690 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a7aac-2a39-407d-ae9b-0eb63d88ff3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:18.544597 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-096a7aac-2a39-407d-ae9b-0eb63d88ff3d None None] MFA Rules not processed for user `83f999f98b0b4cd6968bb24187a9f772`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:18.545903 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-096a7aac-2a39-407d-ae9b-0eb63d88ff3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:18.610430 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1235/2456] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:18.615338 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2307e9b-3606-4a21-a85c-e10b4c83ef35 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:18.674342 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2307e9b-3606-4a21-a85c-e10b4c83ef35 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:18.679314 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c340b-4736-45f1-94b3-a8dc76162129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:18.679899 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c340b-4736-45f1-94b3-a8dc76162129 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:18.680169 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c340b-4736-45f1-94b3-a8dc76162129 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:18.680431 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c340b-4736-45f1-94b3-a8dc76162129 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:18.681041 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130c340b-4736-45f1-94b3-a8dc76162129 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:18.682929 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130c340b-4736-45f1-94b3-a8dc76162129 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:18.709111 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1222/2457] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:48:18 2022] POST /identity/v3/projects => generated 387 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:48:18.713627 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096a7aac-2a39-407d-ae9b-0eb63d88ff3d None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:18.724215 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096a7aac-2a39-407d-ae9b-0eb63d88ff3d None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:18.780100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c61e7c6-4ad1-42ba-97d0-ae3b52137ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:18.780973 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c61e7c6-4ad1-42ba-97d0-ae3b52137ed4 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:18.781260 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c61e7c6-4ad1-42ba-97d0-ae3b52137ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:18.781523 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c61e7c6-4ad1-42ba-97d0-ae3b52137ed4 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:18.782157 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c61e7c6-4ad1-42ba-97d0-ae3b52137ed4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:18.784056 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c61e7c6-4ad1-42ba-97d0-ae3b52137ed4 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:18.805770 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1236/2458] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:48:18 2022] POST /identity/v3/users => generated 300 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:48:18.809952 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c340b-4736-45f1-94b3-a8dc76162129 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:18.821125 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c340b-4736-45f1-94b3-a8dc76162129 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:18.826004 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2889da1f-1c1e-4bc5-8bed-69a777d2eed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:18.826594 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2889da1f-1c1e-4bc5-8bed-69a777d2eed8 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:18.826867 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2889da1f-1c1e-4bc5-8bed-69a777d2eed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:18.827131 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2889da1f-1c1e-4bc5-8bed-69a777d2eed8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:18.827756 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2889da1f-1c1e-4bc5-8bed-69a777d2eed8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:18.829333 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2889da1f-1c1e-4bc5-8bed-69a777d2eed8 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:18.835444 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:18.843438 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1223/2459] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:48:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:48:18.844667 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c61e7c6-4ad1-42ba-97d0-ae3b52137ed4 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:18.854965 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c61e7c6-4ad1-42ba-97d0-ae3b52137ed4 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:18.860279 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0d4632-5d25-49f9-922b-a167e1145a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:18.861091 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d4632-5d25-49f9-922b-a167e1145a9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:18.861368 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d4632-5d25-49f9-922b-a167e1145a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:18.861641 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0d4632-5d25-49f9-922b-a167e1145a9c admin admin] PATH_INFO: `/v3/projects/4fde5021496a4da4a95d67f89cfd03a2/users/c17b662f33624c60b47802c3dea9dba4/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:18.877230 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0d4632-5d25-49f9-922b-a167e1145a9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fde5021496a4da4a95d67f89cfd03a2, user_id=c17b662f33624c60b47802c3dea9dba4, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:18.880708 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0d4632-5d25-49f9-922b-a167e1145a9c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:18.894192 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1237/2460] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:48:18 2022] PUT /identity/v3/projects/4fde5021496a4da4a95d67f89cfd03a2/users/c17b662f33624c60b47802c3dea9dba4/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:18.898170 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2889da1f-1c1e-4bc5-8bed-69a777d2eed8 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:18.908329 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2889da1f-1c1e-4bc5-8bed-69a777d2eed8 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:18.964537 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e1771d-74b7-4dc8-9c66-c6dc8ad4cbd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:18.965137 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e1771d-74b7-4dc8-9c66-c6dc8ad4cbd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:18.965413 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e1771d-74b7-4dc8-9c66-c6dc8ad4cbd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:18.965682 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e1771d-74b7-4dc8-9c66-c6dc8ad4cbd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:18.966529 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e1771d-74b7-4dc8-9c66-c6dc8ad4cbd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:18.968105 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e1771d-74b7-4dc8-9c66-c6dc8ad4cbd5 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:18.978226 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1224/2461] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:48:18 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:48:18.982361 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0d4632-5d25-49f9-922b-a167e1145a9c admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:18.993050 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0d4632-5d25-49f9-922b-a167e1145a9c admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:18.997626 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe82a21d-1ddf-48ed-a9e5-37f6afec62e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:18.998307 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82a21d-1ddf-48ed-a9e5-37f6afec62e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:18.998588 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82a21d-1ddf-48ed-a9e5-37f6afec62e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:18.998859 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82a21d-1ddf-48ed-a9e5-37f6afec62e4 admin admin] PATH_INFO: `/v3/projects/4fde5021496a4da4a95d67f89cfd03a2/users/c17b662f33624c60b47802c3dea9dba4/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:19.000790 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe82a21d-1ddf-48ed-a9e5-37f6afec62e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fde5021496a4da4a95d67f89cfd03a2, user_id=c17b662f33624c60b47802c3dea9dba4, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:19.003446 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe82a21d-1ddf-48ed-a9e5-37f6afec62e4 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:19.013361 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1238/2462] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:48:18 2022] PUT /identity/v3/projects/4fde5021496a4da4a95d67f89cfd03a2/users/c17b662f33624c60b47802c3dea9dba4/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:19.024364 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd651ed-50f4-4165-9029-d4fa10c89277 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:19.024737 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd651ed-50f4-4165-9029-d4fa10c89277 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:19.025017 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd651ed-50f4-4165-9029-d4fa10c89277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:19.053055 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-5cd651ed-50f4-4165-9029-d4fa10c89277 None None] MFA Rules not processed for user `c17b662f33624c60b47802c3dea9dba4`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:19.054436 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-5cd651ed-50f4-4165-9029-d4fa10c89277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:19.122306 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1225/2463] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:19 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:48:19.132826 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe82a21d-1ddf-48ed-a9e5-37f6afec62e4 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:19.134558 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-fe82a21d-1ddf-48ed-a9e5-37f6afec62e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:19.174478 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:19.193257 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe82a21d-1ddf-48ed-a9e5-37f6afec62e4 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:19.200960 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b739f6-5e75-48ce-8e16-36020ac6a39e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:19.201566 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b739f6-5e75-48ce-8e16-36020ac6a39e service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:19.201870 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b739f6-5e75-48ce-8e16-36020ac6a39e service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:19.202156 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b739f6-5e75-48ce-8e16-36020ac6a39e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:19.211397 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b739f6-5e75-48ce-8e16-36020ac6a39e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:19.212832 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b739f6-5e75-48ce-8e16-36020ac6a39e service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:19.222181 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1239/2464] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:48:19 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:48:19.358339 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd651ed-50f4-4165-9029-d4fa10c89277 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:19.419413 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd651ed-50f4-4165-9029-d4fa10c89277 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:19.423747 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7310275-5877-4429-9be4-8a2978fe36af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:19.424363 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7310275-5877-4429-9be4-8a2978fe36af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:19.424635 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7310275-5877-4429-9be4-8a2978fe36af admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:19.424901 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7310275-5877-4429-9be4-8a2978fe36af admin admin] PATH_INFO: `/v3/users/83f999f98b0b4cd6968bb24187a9f772` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:19.426024 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7310275-5877-4429-9be4-8a2978fe36af admin admin] RBAC: Authorizing `identity:delete_user(user_id=83f999f98b0b4cd6968bb24187a9f772)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:19.428073 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7310275-5877-4429-9be4-8a2978fe36af admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:19.449709 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:19.475877 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a7310275-5877-4429-9be4-8a2978fe36af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83f999f98b0b4cd6968bb24187a9f772'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:19.482790 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a7310275-5877-4429-9be4-8a2978fe36af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83f999f98b0b4cd6968bb24187a9f772'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:19.483445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a7310275-5877-4429-9be4-8a2978fe36af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83f999f98b0b4cd6968bb24187a9f772'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:19.491521 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a7310275-5877-4429-9be4-8a2978fe36af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83f999f98b0b4cd6968bb24187a9f772'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:19.492232 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-a7310275-5877-4429-9be4-8a2978fe36af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83f999f98b0b4cd6968bb24187a9f772'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:19.509346 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1226/2465] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:48:19 2022] DELETE /identity/v3/users/83f999f98b0b4cd6968bb24187a9f772 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:19.514243 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ef1f7-d984-4f27-9c5b-f2cfb3b6a8e6 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:19.514736 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ef1f7-d984-4f27-9c5b-f2cfb3b6a8e6 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:19.514821 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ef1f7-d984-4f27-9c5b-f2cfb3b6a8e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:19.534936 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-ea1ef1f7-d984-4f27-9c5b-f2cfb3b6a8e6 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:19.536779 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ea1ef1f7-d984-4f27-9c5b-f2cfb3b6a8e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:19.630773 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1240/2466] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:19.642840 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7310275-5877-4429-9be4-8a2978fe36af admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:19.644273 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a7310275-5877-4429-9be4-8a2978fe36af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:19.701450 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7310275-5877-4429-9be4-8a2978fe36af admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:19.705191 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb3d183-d817-437a-a993-82e8c392f63c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:19.705784 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb3d183-d817-437a-a993-82e8c392f63c service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:19.706083 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb3d183-d817-437a-a993-82e8c392f63c service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:19.706348 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb3d183-d817-437a-a993-82e8c392f63c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:19.716335 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb3d183-d817-437a-a993-82e8c392f63c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:19.717761 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb3d183-d817-437a-a993-82e8c392f63c service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:19.728048 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1227/2467] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:48:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:48:20.025318 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1ef1f7-d984-4f27-9c5b-f2cfb3b6a8e6 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:20.026905 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ea1ef1f7-d984-4f27-9c5b-f2cfb3b6a8e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:20.058309 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1ef1f7-d984-4f27-9c5b-f2cfb3b6a8e6 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:20.064041 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ae000-d435-4fb1-ae9f-92e5bfb22f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:20.064817 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ae000-d435-4fb1-ae9f-92e5bfb22f23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:20.065119 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ae000-d435-4fb1-ae9f-92e5bfb22f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:20.065404 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ae000-d435-4fb1-ae9f-92e5bfb22f23 admin admin] PATH_INFO: `/v3/projects/d2292df20aa847ff874c1afef13b917c` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:20.066754 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6ae000-d435-4fb1-ae9f-92e5bfb22f23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2292df20aa847ff874c1afef13b917c)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:20.068993 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6ae000-d435-4fb1-ae9f-92e5bfb22f23 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:20.118309 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4b6ae000-d435-4fb1-ae9f-92e5bfb22f23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2292df20aa847ff874c1afef13b917c', 'request_id': 'req-4b6ae000-d435-4fb1-ae9f-92e5bfb22f23'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:20.124429 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4b6ae000-d435-4fb1-ae9f-92e5bfb22f23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2292df20aa847ff874c1afef13b917c', 'request_id': 'req-4b6ae000-d435-4fb1-ae9f-92e5bfb22f23'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:20.131054 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4b6ae000-d435-4fb1-ae9f-92e5bfb22f23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2292df20aa847ff874c1afef13b917c', 'request_id': 'req-4b6ae000-d435-4fb1-ae9f-92e5bfb22f23'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:20.141122 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4b6ae000-d435-4fb1-ae9f-92e5bfb22f23 admin admin] The token cache is being invalidate because project d2292df20aa847ff874c1afef13b917c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:48:20.141407 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4b6ae000-d435-4fb1-ae9f-92e5bfb22f23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:20.142059 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4b6ae000-d435-4fb1-ae9f-92e5bfb22f23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:20.143517 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1241/2468] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:48:20 2022] DELETE /identity/v3/projects/d2292df20aa847ff874c1afef13b917c => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:20.148231 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb3d183-d817-437a-a993-82e8c392f63c service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:20.157798 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb3d183-d817-437a-a993-82e8c392f63c service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:20.214908 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217fb512-0e97-47e0-acd4-248aa5359f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:20.215537 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217fb512-0e97-47e0-acd4-248aa5359f26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:20.215818 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217fb512-0e97-47e0-acd4-248aa5359f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:20.216091 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217fb512-0e97-47e0-acd4-248aa5359f26 admin admin] PATH_INFO: `/v3/users/c17b662f33624c60b47802c3dea9dba4` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:20.217209 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217fb512-0e97-47e0-acd4-248aa5359f26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c17b662f33624c60b47802c3dea9dba4)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:20.219177 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217fb512-0e97-47e0-acd4-248aa5359f26 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:20.263478 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:20.265115 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-217fb512-0e97-47e0-acd4-248aa5359f26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c17b662f33624c60b47802c3dea9dba4'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:20.278871 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-217fb512-0e97-47e0-acd4-248aa5359f26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c17b662f33624c60b47802c3dea9dba4'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:20.279575 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-217fb512-0e97-47e0-acd4-248aa5359f26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c17b662f33624c60b47802c3dea9dba4'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:20.287492 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-217fb512-0e97-47e0-acd4-248aa5359f26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c17b662f33624c60b47802c3dea9dba4'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:20.288111 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-217fb512-0e97-47e0-acd4-248aa5359f26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c17b662f33624c60b47802c3dea9dba4'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:20.305555 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1228/2469] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:48:20 2022] DELETE /identity/v3/users/c17b662f33624c60b47802c3dea9dba4 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:20.562161 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ae000-d435-4fb1-ae9f-92e5bfb22f23 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:20.563747 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4b6ae000-d435-4fb1-ae9f-92e5bfb22f23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:20.578826 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ae000-d435-4fb1-ae9f-92e5bfb22f23 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:20.641562 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae44805-f906-4894-951a-1c01bdc47381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:20.642234 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae44805-f906-4894-951a-1c01bdc47381 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:20.642518 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae44805-f906-4894-951a-1c01bdc47381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:20.642791 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae44805-f906-4894-951a-1c01bdc47381 admin admin] PATH_INFO: `/v3/projects/4fde5021496a4da4a95d67f89cfd03a2` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:20.643900 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae44805-f906-4894-951a-1c01bdc47381 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fde5021496a4da4a95d67f89cfd03a2)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:20.645906 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae44805-f906-4894-951a-1c01bdc47381 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:20.648218 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:20.696917 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6ae44805-f906-4894-951a-1c01bdc47381 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fde5021496a4da4a95d67f89cfd03a2', 'request_id': 'req-6ae44805-f906-4894-951a-1c01bdc47381'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:20.703062 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6ae44805-f906-4894-951a-1c01bdc47381 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fde5021496a4da4a95d67f89cfd03a2', 'request_id': 'req-6ae44805-f906-4894-951a-1c01bdc47381'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:20.711189 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6ae44805-f906-4894-951a-1c01bdc47381 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fde5021496a4da4a95d67f89cfd03a2', 'request_id': 'req-6ae44805-f906-4894-951a-1c01bdc47381'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:20.722383 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6ae44805-f906-4894-951a-1c01bdc47381 admin admin] The token cache is being invalidate because project 4fde5021496a4da4a95d67f89cfd03a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:48:20.722678 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6ae44805-f906-4894-951a-1c01bdc47381 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:20.723884 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6ae44805-f906-4894-951a-1c01bdc47381 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:20.725441 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1242/2470] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:48:20 2022] DELETE /identity/v3/projects/4fde5021496a4da4a95d67f89cfd03a2 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:20.730396 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891eb336-9fe9-4e57-ae07-7065fe8069d4 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:20.730897 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891eb336-9fe9-4e57-ae07-7065fe8069d4 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:20.730997 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891eb336-9fe9-4e57-ae07-7065fe8069d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:20.751269 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-891eb336-9fe9-4e57-ae07-7065fe8069d4 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:20.752477 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-891eb336-9fe9-4e57-ae07-7065fe8069d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:20.821834 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1229/2471] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:48:20.829292 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d25ce-3a83-4ced-b36d-2fa235469366 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:20.829545 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d25ce-3a83-4ced-b36d-2fa235469366 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:20.829819 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d25ce-3a83-4ced-b36d-2fa235469366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:20.850955 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-312d25ce-3a83-4ced-b36d-2fa235469366 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:20.852158 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-312d25ce-3a83-4ced-b36d-2fa235469366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:20.861415 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1243/2472] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:20.866121 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891eb336-9fe9-4e57-ae07-7065fe8069d4 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:20.876219 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891eb336-9fe9-4e57-ae07-7065fe8069d4 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:20.880686 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c82a8-a3ee-4638-b27c-93b62bb54acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:20.881354 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c82a8-a3ee-4638-b27c-93b62bb54acc admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:20.885185 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c82a8-a3ee-4638-b27c-93b62bb54acc admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:20.885185 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c82a8-a3ee-4638-b27c-93b62bb54acc admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:20.885185 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0c82a8-a3ee-4638-b27c-93b62bb54acc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:48:20.885185 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0c82a8-a3ee-4638-b27c-93b62bb54acc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:20.885185 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0c82a8-a3ee-4638-b27c-93b62bb54acc admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:20.901618 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1230/2473] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:48:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:48:20.907040 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312d25ce-3a83-4ced-b36d-2fa235469366 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:20.917746 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312d25ce-3a83-4ced-b36d-2fa235469366 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:20.923296 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38238273-b2db-422e-a9e6-e6a0ccc362d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:20.924086 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38238273-b2db-422e-a9e6-e6a0ccc362d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:20.924400 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38238273-b2db-422e-a9e6-e6a0ccc362d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:20.924748 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38238273-b2db-422e-a9e6-e6a0ccc362d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:20.925557 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38238273-b2db-422e-a9e6-e6a0ccc362d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:20.927602 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38238273-b2db-422e-a9e6-e6a0ccc362d8 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:20.967612 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1244/2474] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:48:20 2022] POST /identity/v3/projects => generated 377 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:48:20.975431 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c82a8-a3ee-4638-b27c-93b62bb54acc admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:20.990508 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c82a8-a3ee-4638-b27c-93b62bb54acc admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:21.069808 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59bba1b-a1e6-496d-a906-fdc7d365bd41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:21.070461 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59bba1b-a1e6-496d-a906-fdc7d365bd41 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:21.070736 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59bba1b-a1e6-496d-a906-fdc7d365bd41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:21.071009 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59bba1b-a1e6-496d-a906-fdc7d365bd41 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:21.071641 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59bba1b-a1e6-496d-a906-fdc7d365bd41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:21.073952 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59bba1b-a1e6-496d-a906-fdc7d365bd41 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:21.108194 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1231/2475] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:48:20 2022] POST /identity/v3/users => generated 289 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:48:21.114006 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38238273-b2db-422e-a9e6-e6a0ccc362d8 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:21.125897 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38238273-b2db-422e-a9e6-e6a0ccc362d8 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:21.131226 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fcffc5-9935-4456-b195-27c84ffbc784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:21.132117 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fcffc5-9935-4456-b195-27c84ffbc784 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:21.132566 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fcffc5-9935-4456-b195-27c84ffbc784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:21.132932 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fcffc5-9935-4456-b195-27c84ffbc784 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:21.133989 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fcffc5-9935-4456-b195-27c84ffbc784 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:21.135555 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fcffc5-9935-4456-b195-27c84ffbc784 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:21.147187 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1245/2476] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:48:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:48:21.151111 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59bba1b-a1e6-496d-a906-fdc7d365bd41 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:21.161762 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59bba1b-a1e6-496d-a906-fdc7d365bd41 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:21.166958 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9ef359-fa78-4d01-9f48-1042c40ceeb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:21.167471 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ef359-fa78-4d01-9f48-1042c40ceeb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:21.167745 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ef359-fa78-4d01-9f48-1042c40ceeb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:21.168042 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ef359-fa78-4d01-9f48-1042c40ceeb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:21.168702 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9ef359-fa78-4d01-9f48-1042c40ceeb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:21.170365 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9ef359-fa78-4d01-9f48-1042c40ceeb7 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:21.181639 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1232/2477] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:48:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:48:21.186185 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fcffc5-9935-4456-b195-27c84ffbc784 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:21.196962 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fcffc5-9935-4456-b195-27c84ffbc784 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:21.201796 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd51538c-c063-4626-924e-15493d421020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:21.202488 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51538c-c063-4626-924e-15493d421020 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:21.202786 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51538c-c063-4626-924e-15493d421020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:21.203059 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51538c-c063-4626-924e-15493d421020 admin admin] PATH_INFO: `/v3/projects/eaa54f0d3c3744f190f74df3aadc20a7/users/5a8d81ae369b4979ad8a4a19d0649ee6/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:21.219436 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd51538c-c063-4626-924e-15493d421020 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaa54f0d3c3744f190f74df3aadc20a7, user_id=5a8d81ae369b4979ad8a4a19d0649ee6, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:21.222159 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd51538c-c063-4626-924e-15493d421020 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:21.237586 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1246/2478] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:48:21 2022] PUT /identity/v3/projects/eaa54f0d3c3744f190f74df3aadc20a7/users/5a8d81ae369b4979ad8a4a19d0649ee6/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:48:21.250281 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05eeb3a-2ee7-4f18-bdc0-c020a5bceb0d None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:21.250520 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05eeb3a-2ee7-4f18-bdc0-c020a5bceb0d None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:21.250715 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05eeb3a-2ee7-4f18-bdc0-c020a5bceb0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:21.282887 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-f05eeb3a-2ee7-4f18-bdc0-c020a5bceb0d None None] MFA Rules not processed for user `5a8d81ae369b4979ad8a4a19d0649ee6`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:21.284094 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-f05eeb3a-2ee7-4f18-bdc0-c020a5bceb0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:21.300704 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:21.349595 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1233/2479] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:21.354142 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd51538c-c063-4626-924e-15493d421020 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:21.414088 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd51538c-c063-4626-924e-15493d421020 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:21.418644 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d85ab2-b9f3-4e52-be28-74284d8caaba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:21.419228 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d85ab2-b9f3-4e52-be28-74284d8caaba admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:21.419505 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d85ab2-b9f3-4e52-be28-74284d8caaba admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:21.419774 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d85ab2-b9f3-4e52-be28-74284d8caaba admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:21.420393 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d85ab2-b9f3-4e52-be28-74284d8caaba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:21.422308 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d85ab2-b9f3-4e52-be28-74284d8caaba admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:21.449088 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1247/2480] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:48:21 2022] POST /identity/v3/projects => generated 377 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:48:21.453068 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05eeb3a-2ee7-4f18-bdc0-c020a5bceb0d None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:21.464066 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05eeb3a-2ee7-4f18-bdc0-c020a5bceb0d None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:21.519695 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5896e7-ea7c-4f0b-b188-a1b9fe83fa17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:21.521103 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5896e7-ea7c-4f0b-b188-a1b9fe83fa17 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:21.521382 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5896e7-ea7c-4f0b-b188-a1b9fe83fa17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:21.521650 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5896e7-ea7c-4f0b-b188-a1b9fe83fa17 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:21.522303 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5896e7-ea7c-4f0b-b188-a1b9fe83fa17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:21.524209 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5896e7-ea7c-4f0b-b188-a1b9fe83fa17 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:21.545315 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1234/2481] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:48:21 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:48:21.549350 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d85ab2-b9f3-4e52-be28-74284d8caaba admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:21.560354 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d85ab2-b9f3-4e52-be28-74284d8caaba admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:21.565103 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fe54af-019b-443f-b3c1-6f801ed63158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:21.565701 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe54af-019b-443f-b3c1-6f801ed63158 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:21.566016 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe54af-019b-443f-b3c1-6f801ed63158 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:21.566290 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe54af-019b-443f-b3c1-6f801ed63158 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:21.567446 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fe54af-019b-443f-b3c1-6f801ed63158 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:21.569044 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fe54af-019b-443f-b3c1-6f801ed63158 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:21.579613 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1248/2482] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:48:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:48:21.584374 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5896e7-ea7c-4f0b-b188-a1b9fe83fa17 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:21.594886 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5896e7-ea7c-4f0b-b188-a1b9fe83fa17 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:21.599628 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea910781-11ed-47b1-a74a-ac6ee65356ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:21.600212 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea910781-11ed-47b1-a74a-ac6ee65356ef admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:21.600482 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea910781-11ed-47b1-a74a-ac6ee65356ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:21.600747 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea910781-11ed-47b1-a74a-ac6ee65356ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:21.601403 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea910781-11ed-47b1-a74a-ac6ee65356ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:21.602999 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea910781-11ed-47b1-a74a-ac6ee65356ef admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:21.612958 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1235/2483] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:48:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:48:21.617762 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fe54af-019b-443f-b3c1-6f801ed63158 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:21.623205 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:21.629329 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fe54af-019b-443f-b3c1-6f801ed63158 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:21.635305 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad4dfd8-270e-4a68-b7d1-248488901149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:21.635940 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad4dfd8-270e-4a68-b7d1-248488901149 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:21.636215 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad4dfd8-270e-4a68-b7d1-248488901149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:21.636482 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad4dfd8-270e-4a68-b7d1-248488901149 admin admin] PATH_INFO: `/v3/projects/89c80e87f58b44e5864ab31cf10863f5/users/babd401025804e0ba42952ba881b5850/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:21.650559 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad4dfd8-270e-4a68-b7d1-248488901149 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89c80e87f58b44e5864ab31cf10863f5, user_id=babd401025804e0ba42952ba881b5850, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:21.653743 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad4dfd8-270e-4a68-b7d1-248488901149 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:21.666085 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1249/2484] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:48:21 2022] PUT /identity/v3/projects/89c80e87f58b44e5864ab31cf10863f5/users/babd401025804e0ba42952ba881b5850/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:21.677348 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6661eaaa-af04-40e5-9d9d-fb7d054d07a8 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:21.677627 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6661eaaa-af04-40e5-9d9d-fb7d054d07a8 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:21.677917 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6661eaaa-af04-40e5-9d9d-fb7d054d07a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:21.705870 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-6661eaaa-af04-40e5-9d9d-fb7d054d07a8 None None] MFA Rules not processed for user `babd401025804e0ba42952ba881b5850`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:21.707172 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-6661eaaa-af04-40e5-9d9d-fb7d054d07a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:21.776648 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1236/2485] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:21.781821 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad4dfd8-270e-4a68-b7d1-248488901149 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:21.844360 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad4dfd8-270e-4a68-b7d1-248488901149 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:21.849001 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ea3550-3ac9-419b-bb16-ab49e1a1f246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:21.849594 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea3550-3ac9-419b-bb16-ab49e1a1f246 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:21.849897 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea3550-3ac9-419b-bb16-ab49e1a1f246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:21.850173 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea3550-3ac9-419b-bb16-ab49e1a1f246 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:21.850788 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ea3550-3ac9-419b-bb16-ab49e1a1f246 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:21.852668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ea3550-3ac9-419b-bb16-ab49e1a1f246 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:21.878340 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1250/2486] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:48:21 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:48:21.882989 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6661eaaa-af04-40e5-9d9d-fb7d054d07a8 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:21.893338 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6661eaaa-af04-40e5-9d9d-fb7d054d07a8 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:21.916720 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:21.950467 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c453fe06-1a19-4853-b3ca-40edde184406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:21.951074 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453fe06-1a19-4853-b3ca-40edde184406 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:21.951348 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453fe06-1a19-4853-b3ca-40edde184406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:21.951616 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453fe06-1a19-4853-b3ca-40edde184406 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:21.952241 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c453fe06-1a19-4853-b3ca-40edde184406 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:21.954161 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c453fe06-1a19-4853-b3ca-40edde184406 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:21.976576 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1237/2487] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:48:21 2022] POST /identity/v3/users => generated 295 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:48:21.980705 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ea3550-3ac9-419b-bb16-ab49e1a1f246 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:21.991848 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ea3550-3ac9-419b-bb16-ab49e1a1f246 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:21.996675 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9058aa-f934-462c-9114-8bc3017a63ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:21.997260 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9058aa-f934-462c-9114-8bc3017a63ba admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:21.997543 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9058aa-f934-462c-9114-8bc3017a63ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:21.997822 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9058aa-f934-462c-9114-8bc3017a63ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:21.998586 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9058aa-f934-462c-9114-8bc3017a63ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:22.000235 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9058aa-f934-462c-9114-8bc3017a63ba admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:22.010798 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1251/2488] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:48:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:48:22.014939 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c453fe06-1a19-4853-b3ca-40edde184406 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:22.024669 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c453fe06-1a19-4853-b3ca-40edde184406 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:22.029269 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5530e406-65e4-47df-a4ba-0fae4c2417e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:22.029862 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530e406-65e4-47df-a4ba-0fae4c2417e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:22.030153 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530e406-65e4-47df-a4ba-0fae4c2417e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:22.030422 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530e406-65e4-47df-a4ba-0fae4c2417e4 admin admin] PATH_INFO: `/v3/projects/1fdb36968d444dfb8a47c0cc680b5428/users/4e2817e414004ff6b47ee0bf469359b3/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:22.045424 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5530e406-65e4-47df-a4ba-0fae4c2417e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fdb36968d444dfb8a47c0cc680b5428, user_id=4e2817e414004ff6b47ee0bf469359b3, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:22.048076 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5530e406-65e4-47df-a4ba-0fae4c2417e4 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:22.061077 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1238/2489] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:48:22 2022] PUT /identity/v3/projects/1fdb36968d444dfb8a47c0cc680b5428/users/4e2817e414004ff6b47ee0bf469359b3/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:22.065186 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9058aa-f934-462c-9114-8bc3017a63ba admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:22.075100 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9058aa-f934-462c-9114-8bc3017a63ba admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:22.132072 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b404c64c-69c8-4ada-8be9-8db479d6c97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:22.132739 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b404c64c-69c8-4ada-8be9-8db479d6c97a admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:22.133095 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b404c64c-69c8-4ada-8be9-8db479d6c97a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:22.133375 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b404c64c-69c8-4ada-8be9-8db479d6c97a admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:22.134240 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b404c64c-69c8-4ada-8be9-8db479d6c97a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:22.135055 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5530e406-65e4-47df-a4ba-0fae4c2417e4 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:22.135897 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b404c64c-69c8-4ada-8be9-8db479d6c97a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:22.136825 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-5530e406-65e4-47df-a4ba-0fae4c2417e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:22.146479 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1252/2490] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:48:22 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:48:22.150073 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b404c64c-69c8-4ada-8be9-8db479d6c97a admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:22.152937 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5530e406-65e4-47df-a4ba-0fae4c2417e4 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:22.157598 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb7e85a-1cd5-4398-97ba-6138ff4a7211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:22.158427 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb7e85a-1cd5-4398-97ba-6138ff4a7211 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:22.158587 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb7e85a-1cd5-4398-97ba-6138ff4a7211 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:22.158862 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb7e85a-1cd5-4398-97ba-6138ff4a7211 admin admin] PATH_INFO: `/v3/users/a95e88a42d784991b7e9926e764bc429` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:22.159743 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b404c64c-69c8-4ada-8be9-8db479d6c97a admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:22.159998 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb7e85a-1cd5-4398-97ba-6138ff4a7211 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a95e88a42d784991b7e9926e764bc429)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:22.162436 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb7e85a-1cd5-4398-97ba-6138ff4a7211 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:22.164935 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3cacc9-5b13-4d7a-b053-61491a29a92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:22.165573 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3cacc9-5b13-4d7a-b053-61491a29a92a admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:22.165869 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3cacc9-5b13-4d7a-b053-61491a29a92a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:22.166143 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3cacc9-5b13-4d7a-b053-61491a29a92a admin admin] PATH_INFO: `/v3/projects/1fdb36968d444dfb8a47c0cc680b5428/users/4e2817e414004ff6b47ee0bf469359b3/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:22.168050 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3cacc9-5b13-4d7a-b053-61491a29a92a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fdb36968d444dfb8a47c0cc680b5428, user_id=4e2817e414004ff6b47ee0bf469359b3, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:22.170727 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3cacc9-5b13-4d7a-b053-61491a29a92a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:22.184497 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1253/2491] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:48:22 2022] PUT /identity/v3/projects/1fdb36968d444dfb8a47c0cc680b5428/users/4e2817e414004ff6b47ee0bf469359b3/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:48:22.195730 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8196016a-94b4-4d84-b0f6-da4c6875e041 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:22.201642 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8196016a-94b4-4d84-b0f6-da4c6875e041 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:22.201642 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8196016a-94b4-4d84-b0f6-da4c6875e041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:22.229331 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-bfb7e85a-1cd5-4398-97ba-6138ff4a7211 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a95e88a42d784991b7e9926e764bc429'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:22.230921 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-8196016a-94b4-4d84-b0f6-da4c6875e041 None None] MFA Rules not processed for user `4e2817e414004ff6b47ee0bf469359b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:22.232147 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8196016a-94b4-4d84-b0f6-da4c6875e041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:22.237397 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-bfb7e85a-1cd5-4398-97ba-6138ff4a7211 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a95e88a42d784991b7e9926e764bc429'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:22.238120 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-bfb7e85a-1cd5-4398-97ba-6138ff4a7211 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a95e88a42d784991b7e9926e764bc429'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:22.246797 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-bfb7e85a-1cd5-4398-97ba-6138ff4a7211 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a95e88a42d784991b7e9926e764bc429'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:22.247525 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-bfb7e85a-1cd5-4398-97ba-6138ff4a7211 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a95e88a42d784991b7e9926e764bc429'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:22.280134 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1239/2492] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:48:22 2022] DELETE /identity/v3/users/a95e88a42d784991b7e9926e764bc429 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:48:22.280435 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:22.280541 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf107aad-823a-4929-9b69-37f803d42aa3 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:22.280541 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf107aad-823a-4929-9b69-37f803d42aa3 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:22.280541 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf107aad-823a-4929-9b69-37f803d42aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:22.303915 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-bf107aad-823a-4929-9b69-37f803d42aa3 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:22.305591 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-bf107aad-823a-4929-9b69-37f803d42aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:22.328347 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1254/2493] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:22 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:48:22.341230 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8196016a-94b4-4d84-b0f6-da4c6875e041 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:22.342840 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8196016a-94b4-4d84-b0f6-da4c6875e041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:22.374305 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1240/2494] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:48:22.385003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf107aad-823a-4929-9b69-37f803d42aa3 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:22.386438 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-bf107aad-823a-4929-9b69-37f803d42aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:22.394552 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:22.408354 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8196016a-94b4-4d84-b0f6-da4c6875e041 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:22.459133 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf107aad-823a-4929-9b69-37f803d42aa3 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:22.462911 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d77a4-373c-4bc2-8531-ed43797b1862 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:22.463588 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d77a4-373c-4bc2-8531-ed43797b1862 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:22.463933 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d77a4-373c-4bc2-8531-ed43797b1862 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:22.464267 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d77a4-373c-4bc2-8531-ed43797b1862 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:22.465186 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e98a0-76a6-4baa-bb75-ea8b7d643d32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:22.465802 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e98a0-76a6-4baa-bb75-ea8b7d643d32 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:22.466116 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e98a0-76a6-4baa-bb75-ea8b7d643d32 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:22.466391 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e98a0-76a6-4baa-bb75-ea8b7d643d32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:22.477173 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00d77a4-373c-4bc2-8531-ed43797b1862 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:22.478762 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00d77a4-373c-4bc2-8531-ed43797b1862 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:22.489822 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1241/2495] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:48:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:48:22.542254 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17e98a0-76a6-4baa-bb75-ea8b7d643d32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:22.543502 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17e98a0-76a6-4baa-bb75-ea8b7d643d32 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:22.554245 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1255/2496] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:48:22 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:48:22.602538 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d77a4-373c-4bc2-8531-ed43797b1862 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:22.604039 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e00d77a4-373c-4bc2-8531-ed43797b1862 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:22.616556 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d77a4-373c-4bc2-8531-ed43797b1862 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:22.620877 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73cf4cd-bccf-4bef-8d2f-a24f48fe6a13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:22.621691 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73cf4cd-bccf-4bef-8d2f-a24f48fe6a13 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:22.622000 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73cf4cd-bccf-4bef-8d2f-a24f48fe6a13 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:22.622266 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73cf4cd-bccf-4bef-8d2f-a24f48fe6a13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:22.624050 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d73cf4cd-bccf-4bef-8d2f-a24f48fe6a13 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:22.690058 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73cf4cd-bccf-4bef-8d2f-a24f48fe6a13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:22.691544 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73cf4cd-bccf-4bef-8d2f-a24f48fe6a13 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:22.700727 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1242/2497] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:48:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:48:22.784811 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e98a0-76a6-4baa-bb75-ea8b7d643d32 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:22.786324 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f17e98a0-76a6-4baa-bb75-ea8b7d643d32 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:22.802983 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e98a0-76a6-4baa-bb75-ea8b7d643d32 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:22.809078 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7d580c-11ea-404f-9fd2-ab3abd102e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:22.809794 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d580c-11ea-404f-9fd2-ab3abd102e3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:22.810392 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d580c-11ea-404f-9fd2-ab3abd102e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:22.810737 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d580c-11ea-404f-9fd2-ab3abd102e3e admin admin] PATH_INFO: `/v3/projects/4dc4eb2d576c47d39e3f0fdb80b2c611` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:22.811890 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7d580c-11ea-404f-9fd2-ab3abd102e3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4dc4eb2d576c47d39e3f0fdb80b2c611)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:22.813796 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7d580c-11ea-404f-9fd2-ab3abd102e3e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:22.839975 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:22.864809 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-0b7d580c-11ea-404f-9fd2-ab3abd102e3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4dc4eb2d576c47d39e3f0fdb80b2c611', 'request_id': 'req-0b7d580c-11ea-404f-9fd2-ab3abd102e3e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:22.871130 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-0b7d580c-11ea-404f-9fd2-ab3abd102e3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4dc4eb2d576c47d39e3f0fdb80b2c611', 'request_id': 'req-0b7d580c-11ea-404f-9fd2-ab3abd102e3e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:22.878324 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-0b7d580c-11ea-404f-9fd2-ab3abd102e3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4dc4eb2d576c47d39e3f0fdb80b2c611', 'request_id': 'req-0b7d580c-11ea-404f-9fd2-ab3abd102e3e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:22.890975 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-0b7d580c-11ea-404f-9fd2-ab3abd102e3e admin admin] The token cache is being invalidate because project 4dc4eb2d576c47d39e3f0fdb80b2c611 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:48:22.891267 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-0b7d580c-11ea-404f-9fd2-ab3abd102e3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:22.891921 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-0b7d580c-11ea-404f-9fd2-ab3abd102e3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:22.893475 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1256/2498] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:48:22 2022] DELETE /identity/v3/projects/4dc4eb2d576c47d39e3f0fdb80b2c611 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:22.899007 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99309bda-8171-4ee0-98bd-557fd22d7c6a None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:22.899605 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99309bda-8171-4ee0-98bd-557fd22d7c6a None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:22.899605 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99309bda-8171-4ee0-98bd-557fd22d7c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:22.919019 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-99309bda-8171-4ee0-98bd-557fd22d7c6a None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:22.920210 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-99309bda-8171-4ee0-98bd-557fd22d7c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:22.982228 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1243/2499] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:22.991543 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af07497-c702-4062-a931-33897307d512 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:22.991908 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af07497-c702-4062-a931-33897307d512 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:22.992256 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af07497-c702-4062-a931-33897307d512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:23.030119 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-8af07497-c702-4062-a931-33897307d512 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:23.031501 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8af07497-c702-4062-a931-33897307d512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:23.040298 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1257/2500] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:48:23.044555 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99309bda-8171-4ee0-98bd-557fd22d7c6a None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:23.054933 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99309bda-8171-4ee0-98bd-557fd22d7c6a None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:23.073371 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c947427-ee23-44d6-9c9c-2028bf5ed27a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:23.073371 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c947427-ee23-44d6-9c9c-2028bf5ed27a admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:23.073371 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c947427-ee23-44d6-9c9c-2028bf5ed27a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:23.073371 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c947427-ee23-44d6-9c9c-2028bf5ed27a admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:23.074173 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c947427-ee23-44d6-9c9c-2028bf5ed27a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:48:23.074173 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c947427-ee23-44d6-9c9c-2028bf5ed27a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:23.074173 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c947427-ee23-44d6-9c9c-2028bf5ed27a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:23.079564 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1244/2501] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:48:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:48:23.084649 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af07497-c702-4062-a931-33897307d512 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:23.094408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af07497-c702-4062-a931-33897307d512 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:23.099092 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabc1295-c141-4cda-b850-d1da8ca61a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:23.099797 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc1295-c141-4cda-b850-d1da8ca61a8e admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:23.100070 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc1295-c141-4cda-b850-d1da8ca61a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:23.100342 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc1295-c141-4cda-b850-d1da8ca61a8e admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:23.100955 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabc1295-c141-4cda-b850-d1da8ca61a8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:23.102957 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabc1295-c141-4cda-b850-d1da8ca61a8e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:23.131447 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1258/2502] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:48:23 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:48:23.135556 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c947427-ee23-44d6-9c9c-2028bf5ed27a admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:23.145330 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c947427-ee23-44d6-9c9c-2028bf5ed27a admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:23.174308 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:23.202946 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ea0d91-6100-43a5-9b70-1a4618f09850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:23.203550 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea0d91-6100-43a5-9b70-1a4618f09850 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:23.203822 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea0d91-6100-43a5-9b70-1a4618f09850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:23.204112 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea0d91-6100-43a5-9b70-1a4618f09850 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:23.204736 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ea0d91-6100-43a5-9b70-1a4618f09850 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:23.206658 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ea0d91-6100-43a5-9b70-1a4618f09850 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:23.227052 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1245/2503] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:48:23 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:48:23.231434 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabc1295-c141-4cda-b850-d1da8ca61a8e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:23.242317 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabc1295-c141-4cda-b850-d1da8ca61a8e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:23.246910 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9355f74-24fa-4913-ba3e-725718723075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:23.247501 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9355f74-24fa-4913-ba3e-725718723075 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:23.247781 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9355f74-24fa-4913-ba3e-725718723075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:23.248052 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9355f74-24fa-4913-ba3e-725718723075 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:23.248683 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9355f74-24fa-4913-ba3e-725718723075 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:23.250318 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9355f74-24fa-4913-ba3e-725718723075 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:23.260746 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1259/2504] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:48:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:48:23.264750 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ea0d91-6100-43a5-9b70-1a4618f09850 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:23.274534 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ea0d91-6100-43a5-9b70-1a4618f09850 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:23.279363 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90bb73b-1a3b-43de-a690-32ce37356a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:23.279962 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90bb73b-1a3b-43de-a690-32ce37356a7d admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:23.280235 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90bb73b-1a3b-43de-a690-32ce37356a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:23.280501 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90bb73b-1a3b-43de-a690-32ce37356a7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:23.281162 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90bb73b-1a3b-43de-a690-32ce37356a7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:23.282846 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90bb73b-1a3b-43de-a690-32ce37356a7d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:23.293006 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1246/2505] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:48:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:48:23.297395 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9355f74-24fa-4913-ba3e-725718723075 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:23.307850 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9355f74-24fa-4913-ba3e-725718723075 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:23.312441 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378f031b-4cd2-4e86-960e-ee4adb7e9fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:23.313076 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f031b-4cd2-4e86-960e-ee4adb7e9fa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:23.313408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f031b-4cd2-4e86-960e-ee4adb7e9fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:23.313679 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f031b-4cd2-4e86-960e-ee4adb7e9fa7 admin admin] PATH_INFO: `/v3/projects/ed1bf756bfcb4d52bd82979559f03a20/users/55a6ab5102034cc68a60218d0d434f25/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:23.329730 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378f031b-4cd2-4e86-960e-ee4adb7e9fa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed1bf756bfcb4d52bd82979559f03a20, user_id=55a6ab5102034cc68a60218d0d434f25, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:23.332387 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378f031b-4cd2-4e86-960e-ee4adb7e9fa7 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:23.344347 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1260/2506] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:48:23 2022] PUT /identity/v3/projects/ed1bf756bfcb4d52bd82979559f03a20/users/55a6ab5102034cc68a60218d0d434f25/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:23.349341 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31766743-8e59-4475-b2fd-3b01a7873338 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:23.349644 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31766743-8e59-4475-b2fd-3b01a7873338 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:23.349960 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31766743-8e59-4475-b2fd-3b01a7873338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:23.369253 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-31766743-8e59-4475-b2fd-3b01a7873338 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:23.370475 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-31766743-8e59-4475-b2fd-3b01a7873338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:23.427442 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1247/2507] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:23.437433 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378f031b-4cd2-4e86-960e-ee4adb7e9fa7 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:23.439164 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-378f031b-4cd2-4e86-960e-ee4adb7e9fa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:23.494641 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378f031b-4cd2-4e86-960e-ee4adb7e9fa7 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:23.498237 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21f8073-a610-4028-8f39-d5a7853f413e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:23.498826 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f8073-a610-4028-8f39-d5a7853f413e service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:23.499101 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f8073-a610-4028-8f39-d5a7853f413e service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:23.499369 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f8073-a610-4028-8f39-d5a7853f413e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:23.508494 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21f8073-a610-4028-8f39-d5a7853f413e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:23.509933 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21f8073-a610-4028-8f39-d5a7853f413e service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:23.519179 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1261/2508] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:48:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:48:25.724768 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31766743-8e59-4475-b2fd-3b01a7873338 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:25.727277 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-31766743-8e59-4475-b2fd-3b01a7873338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:25.741536 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31766743-8e59-4475-b2fd-3b01a7873338 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:25.747312 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0ca3fc-33d3-46a7-9b05-1a67176cda63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:25.748065 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ca3fc-33d3-46a7-9b05-1a67176cda63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:25.748347 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ca3fc-33d3-46a7-9b05-1a67176cda63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:25.748620 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ca3fc-33d3-46a7-9b05-1a67176cda63 admin admin] PATH_INFO: `/v3/users/5a8d81ae369b4979ad8a4a19d0649ee6` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:25.750049 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0ca3fc-33d3-46a7-9b05-1a67176cda63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a8d81ae369b4979ad8a4a19d0649ee6)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:25.752210 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0ca3fc-33d3-46a7-9b05-1a67176cda63 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:25.834539 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-bb0ca3fc-33d3-46a7-9b05-1a67176cda63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a8d81ae369b4979ad8a4a19d0649ee6'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:25.842584 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-bb0ca3fc-33d3-46a7-9b05-1a67176cda63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a8d81ae369b4979ad8a4a19d0649ee6'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:25.843432 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-bb0ca3fc-33d3-46a7-9b05-1a67176cda63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a8d81ae369b4979ad8a4a19d0649ee6'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:25.858871 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-bb0ca3fc-33d3-46a7-9b05-1a67176cda63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a8d81ae369b4979ad8a4a19d0649ee6'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:25.859862 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-bb0ca3fc-33d3-46a7-9b05-1a67176cda63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a8d81ae369b4979ad8a4a19d0649ee6'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:25.881899 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1248/2509] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:48:25 2022] DELETE /identity/v3/users/5a8d81ae369b4979ad8a4a19d0649ee6 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:25.885929 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119bb61b-9dc4-4628-b15c-7fee65f49b7d None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:25.886417 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119bb61b-9dc4-4628-b15c-7fee65f49b7d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:25.886542 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119bb61b-9dc4-4628-b15c-7fee65f49b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:25.907137 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-119bb61b-9dc4-4628-b15c-7fee65f49b7d None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:25.908348 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-119bb61b-9dc4-4628-b15c-7fee65f49b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:25.964305 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:25.981581 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1262/2510] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:25.991296 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0ca3fc-33d3-46a7-9b05-1a67176cda63 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:25.993298 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-bb0ca3fc-33d3-46a7-9b05-1a67176cda63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:26.039331 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:26.087007 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0ca3fc-33d3-46a7-9b05-1a67176cda63 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:26.094784 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241bdc56-d270-453e-a2cd-cbcf044f6e2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:26.095488 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241bdc56-d270-453e-a2cd-cbcf044f6e2c service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:26.095841 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241bdc56-d270-453e-a2cd-cbcf044f6e2c service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:26.096160 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241bdc56-d270-453e-a2cd-cbcf044f6e2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:26.106081 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241bdc56-d270-453e-a2cd-cbcf044f6e2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:26.107518 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241bdc56-d270-453e-a2cd-cbcf044f6e2c service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:26.119384 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1249/2511] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:48:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:48:26.643315 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119bb61b-9dc4-4628-b15c-7fee65f49b7d None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:26.645363 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-119bb61b-9dc4-4628-b15c-7fee65f49b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:26.670441 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119bb61b-9dc4-4628-b15c-7fee65f49b7d None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:26.677107 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfa2057-92dc-4286-9d98-e5a73c6f44e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:26.678043 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa2057-92dc-4286-9d98-e5a73c6f44e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:26.678476 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa2057-92dc-4286-9d98-e5a73c6f44e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:26.678897 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa2057-92dc-4286-9d98-e5a73c6f44e7 admin admin] PATH_INFO: `/v3/projects/eaa54f0d3c3744f190f74df3aadc20a7` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:26.680210 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfa2057-92dc-4286-9d98-e5a73c6f44e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaa54f0d3c3744f190f74df3aadc20a7)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:26.683121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfa2057-92dc-4286-9d98-e5a73c6f44e7 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:26.772742 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9cfa2057-92dc-4286-9d98-e5a73c6f44e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaa54f0d3c3744f190f74df3aadc20a7', 'request_id': 'req-9cfa2057-92dc-4286-9d98-e5a73c6f44e7'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:26.789870 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9cfa2057-92dc-4286-9d98-e5a73c6f44e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaa54f0d3c3744f190f74df3aadc20a7', 'request_id': 'req-9cfa2057-92dc-4286-9d98-e5a73c6f44e7'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:26.802159 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9cfa2057-92dc-4286-9d98-e5a73c6f44e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaa54f0d3c3744f190f74df3aadc20a7', 'request_id': 'req-9cfa2057-92dc-4286-9d98-e5a73c6f44e7'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:26.816928 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9cfa2057-92dc-4286-9d98-e5a73c6f44e7 admin admin] The token cache is being invalidate because project eaa54f0d3c3744f190f74df3aadc20a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:48:26.817150 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9cfa2057-92dc-4286-9d98-e5a73c6f44e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:26.817829 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9cfa2057-92dc-4286-9d98-e5a73c6f44e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:26.820555 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1263/2512] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:48:26 2022] DELETE /identity/v3/projects/eaa54f0d3c3744f190f74df3aadc20a7 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:26.824821 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241bdc56-d270-453e-a2cd-cbcf044f6e2c service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:26.836580 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241bdc56-d270-453e-a2cd-cbcf044f6e2c service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:26.922463 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb4a1ed-6d5f-42ba-b5c1-c9295f17ffc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:26.923636 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4a1ed-6d5f-42ba-b5c1-c9295f17ffc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:26.923636 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4a1ed-6d5f-42ba-b5c1-c9295f17ffc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:26.923636 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4a1ed-6d5f-42ba-b5c1-c9295f17ffc7 admin admin] PATH_INFO: `/v3/users/babd401025804e0ba42952ba881b5850` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:26.924765 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb4a1ed-6d5f-42ba-b5c1-c9295f17ffc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=babd401025804e0ba42952ba881b5850)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:26.928247 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb4a1ed-6d5f-42ba-b5c1-c9295f17ffc7 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:27.042253 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-adb4a1ed-6d5f-42ba-b5c1-c9295f17ffc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'babd401025804e0ba42952ba881b5850'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:27.050825 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-adb4a1ed-6d5f-42ba-b5c1-c9295f17ffc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'babd401025804e0ba42952ba881b5850'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:27.051801 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-adb4a1ed-6d5f-42ba-b5c1-c9295f17ffc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'babd401025804e0ba42952ba881b5850'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:27.067279 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-adb4a1ed-6d5f-42ba-b5c1-c9295f17ffc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'babd401025804e0ba42952ba881b5850'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:27.068036 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-adb4a1ed-6d5f-42ba-b5c1-c9295f17ffc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'babd401025804e0ba42952ba881b5850'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:27.100594 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1250/2513] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:48:26 2022] DELETE /identity/v3/users/babd401025804e0ba42952ba881b5850 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:27.710724 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfa2057-92dc-4286-9d98-e5a73c6f44e7 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:27.712237 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9cfa2057-92dc-4286-9d98-e5a73c6f44e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:27.730485 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfa2057-92dc-4286-9d98-e5a73c6f44e7 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:27.820367 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f06376-ef40-4747-9ae2-1b20a8d2dc08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:27.821288 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f06376-ef40-4747-9ae2-1b20a8d2dc08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:27.821571 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f06376-ef40-4747-9ae2-1b20a8d2dc08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:27.822424 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f06376-ef40-4747-9ae2-1b20a8d2dc08 admin admin] PATH_INFO: `/v3/projects/89c80e87f58b44e5864ab31cf10863f5` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:27.823523 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f06376-ef40-4747-9ae2-1b20a8d2dc08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89c80e87f58b44e5864ab31cf10863f5)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:27.825095 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f06376-ef40-4747-9ae2-1b20a8d2dc08 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:27.881874 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-32f06376-ef40-4747-9ae2-1b20a8d2dc08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89c80e87f58b44e5864ab31cf10863f5', 'request_id': 'req-32f06376-ef40-4747-9ae2-1b20a8d2dc08'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:27.888629 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-32f06376-ef40-4747-9ae2-1b20a8d2dc08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89c80e87f58b44e5864ab31cf10863f5', 'request_id': 'req-32f06376-ef40-4747-9ae2-1b20a8d2dc08'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:27.898417 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-32f06376-ef40-4747-9ae2-1b20a8d2dc08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89c80e87f58b44e5864ab31cf10863f5', 'request_id': 'req-32f06376-ef40-4747-9ae2-1b20a8d2dc08'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:27.914213 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-32f06376-ef40-4747-9ae2-1b20a8d2dc08 admin admin] The token cache is being invalidate because project 89c80e87f58b44e5864ab31cf10863f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:48:27.914501 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-32f06376-ef40-4747-9ae2-1b20a8d2dc08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:27.917110 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-32f06376-ef40-4747-9ae2-1b20a8d2dc08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:27.918937 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1264/2514] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:48:27 2022] DELETE /identity/v3/projects/89c80e87f58b44e5864ab31cf10863f5 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:27.922663 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb4a1ed-6d5f-42ba-b5c1-c9295f17ffc7 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:27.924159 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-adb4a1ed-6d5f-42ba-b5c1-c9295f17ffc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:27.937630 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb4a1ed-6d5f-42ba-b5c1-c9295f17ffc7 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:27.994060 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:28.011115 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51b24e1-d797-4c90-919b-e32f5ac0def1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:28.011754 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51b24e1-d797-4c90-919b-e32f5ac0def1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:28.012034 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51b24e1-d797-4c90-919b-e32f5ac0def1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:28.012297 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51b24e1-d797-4c90-919b-e32f5ac0def1 admin admin] PATH_INFO: `/v3/users/4e2817e414004ff6b47ee0bf469359b3` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:28.013429 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51b24e1-d797-4c90-919b-e32f5ac0def1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e2817e414004ff6b47ee0bf469359b3)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:28.016726 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51b24e1-d797-4c90-919b-e32f5ac0def1 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:28.068019 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d51b24e1-d797-4c90-919b-e32f5ac0def1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e2817e414004ff6b47ee0bf469359b3'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:28.075548 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d51b24e1-d797-4c90-919b-e32f5ac0def1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e2817e414004ff6b47ee0bf469359b3'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:28.075944 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d51b24e1-d797-4c90-919b-e32f5ac0def1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e2817e414004ff6b47ee0bf469359b3'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:28.086675 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d51b24e1-d797-4c90-919b-e32f5ac0def1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e2817e414004ff6b47ee0bf469359b3'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:28.087626 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d51b24e1-d797-4c90-919b-e32f5ac0def1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e2817e414004ff6b47ee0bf469359b3'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:28.113355 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1251/2515] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:48:27 2022] DELETE /identity/v3/users/4e2817e414004ff6b47ee0bf469359b3 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:48:28.877437 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f06376-ef40-4747-9ae2-1b20a8d2dc08 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:28.879154 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-32f06376-ef40-4747-9ae2-1b20a8d2dc08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:28.891295 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:28.906989 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f06376-ef40-4747-9ae2-1b20a8d2dc08 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:29.011244 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdebc66-b0d1-4688-8528-f971f4bdc739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:29.012041 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdebc66-b0d1-4688-8528-f971f4bdc739 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:29.012418 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdebc66-b0d1-4688-8528-f971f4bdc739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:29.014833 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdebc66-b0d1-4688-8528-f971f4bdc739 admin admin] PATH_INFO: `/v3/projects/1fdb36968d444dfb8a47c0cc680b5428` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:29.016125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdebc66-b0d1-4688-8528-f971f4bdc739 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fdb36968d444dfb8a47c0cc680b5428)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:29.019346 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdebc66-b0d1-4688-8528-f971f4bdc739 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:29.081212 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-acdebc66-b0d1-4688-8528-f971f4bdc739 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fdb36968d444dfb8a47c0cc680b5428', 'request_id': 'req-acdebc66-b0d1-4688-8528-f971f4bdc739'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:29.090731 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-acdebc66-b0d1-4688-8528-f971f4bdc739 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fdb36968d444dfb8a47c0cc680b5428', 'request_id': 'req-acdebc66-b0d1-4688-8528-f971f4bdc739'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:29.102755 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-acdebc66-b0d1-4688-8528-f971f4bdc739 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fdb36968d444dfb8a47c0cc680b5428', 'request_id': 'req-acdebc66-b0d1-4688-8528-f971f4bdc739'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:29.117801 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-acdebc66-b0d1-4688-8528-f971f4bdc739 admin admin] The token cache is being invalidate because project 1fdb36968d444dfb8a47c0cc680b5428 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:48:29.118157 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-acdebc66-b0d1-4688-8528-f971f4bdc739 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:29.118972 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-acdebc66-b0d1-4688-8528-f971f4bdc739 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:29.121889 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1265/2516] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:48:28 2022] DELETE /identity/v3/projects/1fdb36968d444dfb8a47c0cc680b5428 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:29.127190 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3b8325-4c8b-4905-a8a6-02d69519f556 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:29.127763 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3b8325-4c8b-4905-a8a6-02d69519f556 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:29.127763 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3b8325-4c8b-4905-a8a6-02d69519f556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:29.155937 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-0f3b8325-4c8b-4905-a8a6-02d69519f556 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:29.157520 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-0f3b8325-4c8b-4905-a8a6-02d69519f556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:29.239217 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1252/2517] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:29.240993 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46389ed-bd37-4002-b02d-786597b0983e None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:29.241281 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46389ed-bd37-4002-b02d-786597b0983e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:29.241564 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46389ed-bd37-4002-b02d-786597b0983e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:29.274209 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-f46389ed-bd37-4002-b02d-786597b0983e None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:29.275841 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f46389ed-bd37-4002-b02d-786597b0983e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:29.289139 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1266/2518] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:48:29.293614 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3b8325-4c8b-4905-a8a6-02d69519f556 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:29.307445 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3b8325-4c8b-4905-a8a6-02d69519f556 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:29.312573 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83acad1b-c0c6-41c5-a50d-eb7570a0cef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:29.313212 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acad1b-c0c6-41c5-a50d-eb7570a0cef0 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:29.313485 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acad1b-c0c6-41c5-a50d-eb7570a0cef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:29.313756 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acad1b-c0c6-41c5-a50d-eb7570a0cef0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:29.314600 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83acad1b-c0c6-41c5-a50d-eb7570a0cef0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:48:29.314967 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83acad1b-c0c6-41c5-a50d-eb7570a0cef0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:29.316812 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83acad1b-c0c6-41c5-a50d-eb7570a0cef0 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:29.333771 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1253/2519] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:48:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:48:29.334809 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46389ed-bd37-4002-b02d-786597b0983e None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:29.344932 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46389ed-bd37-4002-b02d-786597b0983e None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:29.350814 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3479c916-3ad6-466a-8eb8-86df9b7fa6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:29.351617 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479c916-3ad6-466a-8eb8-86df9b7fa6dc admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:29.352273 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479c916-3ad6-466a-8eb8-86df9b7fa6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:29.352690 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479c916-3ad6-466a-8eb8-86df9b7fa6dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:29.353470 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3479c916-3ad6-466a-8eb8-86df9b7fa6dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:29.355856 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3479c916-3ad6-466a-8eb8-86df9b7fa6dc admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:29.391289 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1267/2520] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:48:29 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:48:29.395805 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83acad1b-c0c6-41c5-a50d-eb7570a0cef0 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:29.412285 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83acad1b-c0c6-41c5-a50d-eb7570a0cef0 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:29.471976 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:29.492825 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b3be11-3d09-4544-b9b3-e01468f180cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:29.493763 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b3be11-3d09-4544-b9b3-e01468f180cb admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:29.494184 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b3be11-3d09-4544-b9b3-e01468f180cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:29.494544 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b3be11-3d09-4544-b9b3-e01468f180cb admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:29.495490 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b3be11-3d09-4544-b9b3-e01468f180cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:29.497545 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b3be11-3d09-4544-b9b3-e01468f180cb admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:29.546909 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1254/2521] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:48:29 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:48:29.552078 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3479c916-3ad6-466a-8eb8-86df9b7fa6dc admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:29.568367 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3479c916-3ad6-466a-8eb8-86df9b7fa6dc admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:29.580409 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13676129-fbbd-45a4-ac58-4c314ef5dd26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:29.581045 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13676129-fbbd-45a4-ac58-4c314ef5dd26 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:29.581329 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13676129-fbbd-45a4-ac58-4c314ef5dd26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:29.581607 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13676129-fbbd-45a4-ac58-4c314ef5dd26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:29.582849 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13676129-fbbd-45a4-ac58-4c314ef5dd26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:29.596649 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13676129-fbbd-45a4-ac58-4c314ef5dd26 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:29.620501 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1268/2522] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:48:29 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:48:29.623615 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b3be11-3d09-4544-b9b3-e01468f180cb admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:29.637384 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b3be11-3d09-4544-b9b3-e01468f180cb admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:29.642865 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8189644a-43ef-437c-b330-86f905269642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:29.643484 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8189644a-43ef-437c-b330-86f905269642 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:29.643762 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8189644a-43ef-437c-b330-86f905269642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:29.644036 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8189644a-43ef-437c-b330-86f905269642 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:29.645236 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8189644a-43ef-437c-b330-86f905269642 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:29.646903 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8189644a-43ef-437c-b330-86f905269642 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:29.673570 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1255/2523] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:48:29 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:48:29.674091 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13676129-fbbd-45a4-ac58-4c314ef5dd26 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:29.687659 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13676129-fbbd-45a4-ac58-4c314ef5dd26 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:29.693057 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73694c9-cd55-4aaf-be14-ecd57ffba7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:29.693724 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73694c9-cd55-4aaf-be14-ecd57ffba7f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:29.694046 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73694c9-cd55-4aaf-be14-ecd57ffba7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:29.694322 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73694c9-cd55-4aaf-be14-ecd57ffba7f0 admin admin] PATH_INFO: `/v3/projects/e14714c2f8ba4d1b9b009e829547f1c7/users/48f6632d9fe24aeeb9af6955f85a4c0c/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:29.710990 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73694c9-cd55-4aaf-be14-ecd57ffba7f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e14714c2f8ba4d1b9b009e829547f1c7, user_id=48f6632d9fe24aeeb9af6955f85a4c0c, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:29.714312 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73694c9-cd55-4aaf-be14-ecd57ffba7f0 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:29.733043 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1269/2524] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:48:29 2022] PUT /identity/v3/projects/e14714c2f8ba4d1b9b009e829547f1c7/users/48f6632d9fe24aeeb9af6955f85a4c0c/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:29.739330 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b974d-7729-4eea-8692-58bde412050a None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:29.746775 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b974d-7729-4eea-8692-58bde412050a None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:29.746775 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b974d-7729-4eea-8692-58bde412050a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:29.768567 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a100f149-8e3d-449c-87d3-360bd5197d44 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:29.768961 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a100f149-8e3d-449c-87d3-360bd5197d44 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:29.769392 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a100f149-8e3d-449c-87d3-360bd5197d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:29.774289 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-654b974d-7729-4eea-8692-58bde412050a None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:29.775611 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-654b974d-7729-4eea-8692-58bde412050a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:29.808239 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-a100f149-8e3d-449c-87d3-360bd5197d44 None None] MFA Rules not processed for user `55a6ab5102034cc68a60218d0d434f25`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:29.809482 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a100f149-8e3d-449c-87d3-360bd5197d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:29.827619 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:29.841207 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1256/2525] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:29.850991 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b974d-7729-4eea-8692-58bde412050a None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:29.852696 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-654b974d-7729-4eea-8692-58bde412050a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:29.881448 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1270/2526] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:48:29.885825 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a100f149-8e3d-449c-87d3-360bd5197d44 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:29.887278 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a100f149-8e3d-449c-87d3-360bd5197d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:29.902215 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a100f149-8e3d-449c-87d3-360bd5197d44 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:29.907271 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5159ab4a-5533-473b-a47b-4e7282e86dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:29.907873 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5159ab4a-5533-473b-a47b-4e7282e86dc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:29.908152 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5159ab4a-5533-473b-a47b-4e7282e86dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:29.908419 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5159ab4a-5533-473b-a47b-4e7282e86dc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:29.909040 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5159ab4a-5533-473b-a47b-4e7282e86dc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:29.911014 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5159ab4a-5533-473b-a47b-4e7282e86dc0 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:29.915956 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b974d-7729-4eea-8692-58bde412050a None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:29.919755 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddedfc6-fe6a-4ad9-8bc0-821180b20cf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:29.920365 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddedfc6-fe6a-4ad9-8bc0-821180b20cf5 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:29.920638 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddedfc6-fe6a-4ad9-8bc0-821180b20cf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:29.920906 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddedfc6-fe6a-4ad9-8bc0-821180b20cf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:29.931536 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddedfc6-fe6a-4ad9-8bc0-821180b20cf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:29.933029 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddedfc6-fe6a-4ad9-8bc0-821180b20cf5 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:29.944889 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1257/2527] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:48:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:48:29.945755 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1271/2528] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:48:29 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:48:29.949794 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddedfc6-fe6a-4ad9-8bc0-821180b20cf5 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:29.963005 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddedfc6-fe6a-4ad9-8bc0-821180b20cf5 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:30.049389 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ef6ea-6390-4221-bedc-254fcb6f6bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:30.050117 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ef6ea-6390-4221-bedc-254fcb6f6bb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:30.050401 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ef6ea-6390-4221-bedc-254fcb6f6bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:30.051287 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ef6ea-6390-4221-bedc-254fcb6f6bb7 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:30.051960 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911ef6ea-6390-4221-bedc-254fcb6f6bb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:30.053916 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911ef6ea-6390-4221-bedc-254fcb6f6bb7 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:30.082438 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1258/2529] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:48:29 2022] POST /identity/v3/users => generated 293 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:48:30.086344 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5159ab4a-5533-473b-a47b-4e7282e86dc0 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:30.096987 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5159ab4a-5533-473b-a47b-4e7282e86dc0 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:30.101817 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a50c04-28bd-4f7a-a08c-213380cec7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:30.102571 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a50c04-28bd-4f7a-a08c-213380cec7de admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:30.102857 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a50c04-28bd-4f7a-a08c-213380cec7de admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:30.103129 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a50c04-28bd-4f7a-a08c-213380cec7de admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:30.103781 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a50c04-28bd-4f7a-a08c-213380cec7de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:30.105380 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a50c04-28bd-4f7a-a08c-213380cec7de admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:30.119219 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1272/2530] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:48:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:48:30.124594 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ef6ea-6390-4221-bedc-254fcb6f6bb7 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:30.128590 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:30.133097 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ef6ea-6390-4221-bedc-254fcb6f6bb7 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:30.139125 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cfe338-83c3-428c-addf-decf737738fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:30.139784 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cfe338-83c3-428c-addf-decf737738fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:30.140062 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cfe338-83c3-428c-addf-decf737738fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:30.140332 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cfe338-83c3-428c-addf-decf737738fe admin admin] PATH_INFO: `/v3/projects/dec029c8807c43d6982f9e7fe23c90b2/users/87b8824b42a942b4a7b83d1a10f752ad/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:30.157281 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cfe338-83c3-428c-addf-decf737738fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=dec029c8807c43d6982f9e7fe23c90b2, user_id=87b8824b42a942b4a7b83d1a10f752ad, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:30.159888 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cfe338-83c3-428c-addf-decf737738fe admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:30.185827 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1259/2531] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:48:30 2022] PUT /identity/v3/projects/dec029c8807c43d6982f9e7fe23c90b2/users/87b8824b42a942b4a7b83d1a10f752ad/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:48:30.187043 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a50c04-28bd-4f7a-a08c-213380cec7de admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:30.200408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a50c04-28bd-4f7a-a08c-213380cec7de admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:30.301337 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896f3ca9-c701-40df-9d94-9f46eb974b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:30.302033 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f3ca9-c701-40df-9d94-9f46eb974b4b admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:30.302586 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f3ca9-c701-40df-9d94-9f46eb974b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:30.302872 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f3ca9-c701-40df-9d94-9f46eb974b4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:30.303749 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896f3ca9-c701-40df-9d94-9f46eb974b4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:30.305743 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896f3ca9-c701-40df-9d94-9f46eb974b4b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:30.319417 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1273/2532] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:48:30 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:48:30.324291 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cfe338-83c3-428c-addf-decf737738fe admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:30.338192 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cfe338-83c3-428c-addf-decf737738fe admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:30.342984 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396fa98a-2e99-4df7-9e4b-14eab5cdec62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:30.343804 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396fa98a-2e99-4df7-9e4b-14eab5cdec62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:30.343960 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396fa98a-2e99-4df7-9e4b-14eab5cdec62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:30.344201 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396fa98a-2e99-4df7-9e4b-14eab5cdec62 admin admin] PATH_INFO: `/v3/projects/dec029c8807c43d6982f9e7fe23c90b2/users/87b8824b42a942b4a7b83d1a10f752ad/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:30.346197 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396fa98a-2e99-4df7-9e4b-14eab5cdec62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dec029c8807c43d6982f9e7fe23c90b2, user_id=87b8824b42a942b4a7b83d1a10f752ad, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:30.349316 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396fa98a-2e99-4df7-9e4b-14eab5cdec62 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:30.361582 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1260/2533] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:48:30 2022] PUT /identity/v3/projects/dec029c8807c43d6982f9e7fe23c90b2/users/87b8824b42a942b4a7b83d1a10f752ad/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:48:30.829922 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052fac71-9435-485a-b541-e7e9391fa1da None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:30.830512 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052fac71-9435-485a-b541-e7e9391fa1da None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:30.830819 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052fac71-9435-485a-b541-e7e9391fa1da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:30.868786 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-052fac71-9435-485a-b541-e7e9391fa1da None None] MFA Rules not processed for user `48f6632d9fe24aeeb9af6955f85a4c0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:30.870613 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-052fac71-9435-485a-b541-e7e9391fa1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:30.949044 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1274/2534] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:30.950449 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396fa98a-2e99-4df7-9e4b-14eab5cdec62 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:31.031760 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396fa98a-2e99-4df7-9e4b-14eab5cdec62 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:31.038733 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851e706-c549-4366-93e2-984780dfa06e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:31.039666 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851e706-c549-4366-93e2-984780dfa06e admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:31.040192 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851e706-c549-4366-93e2-984780dfa06e admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:31.040671 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e851e706-c549-4366-93e2-984780dfa06e admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:31.041440 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e851e706-c549-4366-93e2-984780dfa06e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:31.043913 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e851e706-c549-4366-93e2-984780dfa06e admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:31.091871 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1261/2535] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:48:30 2022] POST /identity/v3/projects => generated 371 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:48:31.095887 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052fac71-9435-485a-b541-e7e9391fa1da None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:31.108787 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052fac71-9435-485a-b541-e7e9391fa1da None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:31.119956 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:31.176256 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8c295-851a-4879-9930-1065c67ee934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:31.176873 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8c295-851a-4879-9930-1065c67ee934 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:31.177334 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8c295-851a-4879-9930-1065c67ee934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:31.177612 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8c295-851a-4879-9930-1065c67ee934 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:31.178490 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e8c295-851a-4879-9930-1065c67ee934 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:31.180413 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e8c295-851a-4879-9930-1065c67ee934 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:31.208728 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1275/2536] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:48:31 2022] POST /identity/v3/users => generated 292 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:48:31.212742 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851e706-c549-4366-93e2-984780dfa06e admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:31.224121 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e851e706-c549-4366-93e2-984780dfa06e admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:31.230245 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24037ee5-661b-4a22-9db9-694a08d83ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:31.231068 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24037ee5-661b-4a22-9db9-694a08d83ba4 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:31.231398 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24037ee5-661b-4a22-9db9-694a08d83ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:31.231855 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24037ee5-661b-4a22-9db9-694a08d83ba4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:31.232672 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24037ee5-661b-4a22-9db9-694a08d83ba4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:31.236242 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24037ee5-661b-4a22-9db9-694a08d83ba4 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:31.257893 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1262/2537] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:48:31 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:48:31.260663 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8c295-851a-4879-9930-1065c67ee934 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:31.282923 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8c295-851a-4879-9930-1065c67ee934 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:31.288207 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdce3d9-bf8d-4fba-b563-e336724a1ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:31.289887 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdce3d9-bf8d-4fba-b563-e336724a1ac1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:31.290253 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdce3d9-bf8d-4fba-b563-e336724a1ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:31.297601 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdce3d9-bf8d-4fba-b563-e336724a1ac1 admin admin] PATH_INFO: `/v3/projects/e1d48929f99c4f00aa13fb134d1072a1/users/a3e63ada65b54f1fa28d6a5b6eeb9da3/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:31.350499 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdce3d9-bf8d-4fba-b563-e336724a1ac1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1d48929f99c4f00aa13fb134d1072a1, user_id=a3e63ada65b54f1fa28d6a5b6eeb9da3, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:31.353768 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdce3d9-bf8d-4fba-b563-e336724a1ac1 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:31.374862 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1276/2538] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:48:31 2022] PUT /identity/v3/projects/e1d48929f99c4f00aa13fb134d1072a1/users/a3e63ada65b54f1fa28d6a5b6eeb9da3/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:31.379555 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24037ee5-661b-4a22-9db9-694a08d83ba4 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:31.397242 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24037ee5-661b-4a22-9db9-694a08d83ba4 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:31.458289 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdce3d9-bf8d-4fba-b563-e336724a1ac1 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:31.460726 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0cdce3d9-bf8d-4fba-b563-e336724a1ac1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:31.495331 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89364c12-e12d-45dd-b94a-3625d9a5cd6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:31.495965 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89364c12-e12d-45dd-b94a-3625d9a5cd6e admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:31.496240 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89364c12-e12d-45dd-b94a-3625d9a5cd6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:31.496510 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89364c12-e12d-45dd-b94a-3625d9a5cd6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:31.497643 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89364c12-e12d-45dd-b94a-3625d9a5cd6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:31.500043 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89364c12-e12d-45dd-b94a-3625d9a5cd6e admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:31.513672 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1263/2539] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:48:31 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:48:31.518868 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89364c12-e12d-45dd-b94a-3625d9a5cd6e admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:31.535222 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89364c12-e12d-45dd-b94a-3625d9a5cd6e admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:31.543167 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4402a656-e697-4958-ba80-3b4b64df5940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:31.545605 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4402a656-e697-4958-ba80-3b4b64df5940 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:31.545928 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4402a656-e697-4958-ba80-3b4b64df5940 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:31.546089 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdce3d9-bf8d-4fba-b563-e336724a1ac1 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:31.546213 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4402a656-e697-4958-ba80-3b4b64df5940 admin admin] PATH_INFO: `/v3/projects/e1d48929f99c4f00aa13fb134d1072a1/users/a3e63ada65b54f1fa28d6a5b6eeb9da3/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:31.549569 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4402a656-e697-4958-ba80-3b4b64df5940 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1d48929f99c4f00aa13fb134d1072a1, user_id=a3e63ada65b54f1fa28d6a5b6eeb9da3, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:31.551925 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f7dbb-1994-49e4-a482-e66b004e645a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:31.552649 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f7dbb-1994-49e4-a482-e66b004e645a service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:31.552932 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4402a656-e697-4958-ba80-3b4b64df5940 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:31.553257 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f7dbb-1994-49e4-a482-e66b004e645a service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:31.553604 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f7dbb-1994-49e4-a482-e66b004e645a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:31.556118 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b39f7dbb-1994-49e4-a482-e66b004e645a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:31.572073 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1264/2540] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:48:31 2022] PUT /identity/v3/projects/e1d48929f99c4f00aa13fb134d1072a1/users/a3e63ada65b54f1fa28d6a5b6eeb9da3/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:31.574952 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39f7dbb-1994-49e4-a482-e66b004e645a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:31.584405 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39f7dbb-1994-49e4-a482-e66b004e645a service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:31.604129 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1277/2541] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:48:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:48:32.998477 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9a583-b520-4c22-89ff-fea98d9450a7 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:32.999305 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9a583-b520-4c22-89ff-fea98d9450a7 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:32.999837 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9a583-b520-4c22-89ff-fea98d9450a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:33.064834 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-5cb9a583-b520-4c22-89ff-fea98d9450a7 None None] MFA Rules not processed for user `a3e63ada65b54f1fa28d6a5b6eeb9da3`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:33.066166 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-5cb9a583-b520-4c22-89ff-fea98d9450a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:33.094163 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:33.198037 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1265/2542] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:32 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:33.204510 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f7dbb-1994-49e4-a482-e66b004e645a service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:33.206096 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b39f7dbb-1994-49e4-a482-e66b004e645a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:33.340810 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f7dbb-1994-49e4-a482-e66b004e645a service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:33.459934 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:33.503174 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca83d74c-d644-4a3d-9603-c45c7cc09228 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:33.504279 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83d74c-d644-4a3d-9603-c45c7cc09228 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:33.504802 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83d74c-d644-4a3d-9603-c45c7cc09228 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:33.505259 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83d74c-d644-4a3d-9603-c45c7cc09228 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:33.620232 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca83d74c-d644-4a3d-9603-c45c7cc09228 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:33.622056 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca83d74c-d644-4a3d-9603-c45c7cc09228 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:33.648538 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1278/2543] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:48:33 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:48:35.214933 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb9a583-b520-4c22-89ff-fea98d9450a7 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:35.216389 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-5cb9a583-b520-4c22-89ff-fea98d9450a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:35.256789 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb9a583-b520-4c22-89ff-fea98d9450a7 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:35.283715 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edabc0d9-7e5d-4497-80bd-6cabb4d5a6c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:35.286783 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edabc0d9-7e5d-4497-80bd-6cabb4d5a6c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:35.289247 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edabc0d9-7e5d-4497-80bd-6cabb4d5a6c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:35.289773 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edabc0d9-7e5d-4497-80bd-6cabb4d5a6c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:35.321348 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edabc0d9-7e5d-4497-80bd-6cabb4d5a6c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:35.322989 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edabc0d9-7e5d-4497-80bd-6cabb4d5a6c6 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:35.370504 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1266/2544] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:48:35 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:48:36.912737 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e81a12f-2b64-4de5-b10f-4e87578b5040 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:36.913242 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e81a12f-2b64-4de5-b10f-4e87578b5040 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:36.913346 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e81a12f-2b64-4de5-b10f-4e87578b5040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:37.001506 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-1e81a12f-2b64-4de5-b10f-4e87578b5040 None None] MFA Rules not processed for user `87b8824b42a942b4a7b83d1a10f752ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:37.002766 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1e81a12f-2b64-4de5-b10f-4e87578b5040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:37.086479 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1279/2545] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:36 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:37.101538 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edabc0d9-7e5d-4497-80bd-6cabb4d5a6c6 service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:37.102930 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-edabc0d9-7e5d-4497-80bd-6cabb4d5a6c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:37.126827 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e81a12f-2b64-4de5-b10f-4e87578b5040 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:37.133546 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1e81a12f-2b64-4de5-b10f-4e87578b5040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:37.208393 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edabc0d9-7e5d-4497-80bd-6cabb4d5a6c6 service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:37.213829 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cec3c5-d0bb-494b-b38e-db9105031769 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf14b64bf04b2ba74d2fe1fd41ff71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:37.214491 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cec3c5-d0bb-494b-b38e-db9105031769 service nova] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:37.214959 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cec3c5-d0bb-494b-b38e-db9105031769 service nova] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:37.215431 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cec3c5-d0bb-494b-b38e-db9105031769 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:37.225454 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e81a12f-2b64-4de5-b10f-4e87578b5040 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:37.235196 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884985c8-21fe-4abf-b87b-5d984fcd262c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:37.236021 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884985c8-21fe-4abf-b87b-5d984fcd262c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:37.236399 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884985c8-21fe-4abf-b87b-5d984fcd262c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:37.236790 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884985c8-21fe-4abf-b87b-5d984fcd262c admin admin] PATH_INFO: `/v3/users/f22b26ed7a47480e8448175517170172` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:37.238446 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884985c8-21fe-4abf-b87b-5d984fcd262c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f22b26ed7a47480e8448175517170172)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:37.240722 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884985c8-21fe-4abf-b87b-5d984fcd262c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:37.278241 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:37.321404 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cec3c5-d0bb-494b-b38e-db9105031769 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:37.328030 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cec3c5-d0bb-494b-b38e-db9105031769 service nova] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:37.332189 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-884985c8-21fe-4abf-b87b-5d984fcd262c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f22b26ed7a47480e8448175517170172'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:37.332937 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-884985c8-21fe-4abf-b87b-5d984fcd262c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f22b26ed7a47480e8448175517170172'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:37.345887 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1267/2546] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:48:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:48:37.352199 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-884985c8-21fe-4abf-b87b-5d984fcd262c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f22b26ed7a47480e8448175517170172'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:37.362420 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-884985c8-21fe-4abf-b87b-5d984fcd262c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f22b26ed7a47480e8448175517170172'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:37.374159 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-884985c8-21fe-4abf-b87b-5d984fcd262c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f22b26ed7a47480e8448175517170172'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:37.381120 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1280/2547] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:48:37 2022] DELETE /identity/v3/users/f22b26ed7a47480e8448175517170172 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:48:37.605963 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cec3c5-d0bb-494b-b38e-db9105031769 service nova] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:37.626788 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cec3c5-d0bb-494b-b38e-db9105031769 service nova] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:37.673335 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:37.719179 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0485c8f-606e-4d86-a080-28e86886a277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:37.720064 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0485c8f-606e-4d86-a080-28e86886a277 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:37.720407 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0485c8f-606e-4d86-a080-28e86886a277 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:37.720776 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0485c8f-606e-4d86-a080-28e86886a277 admin admin] PATH_INFO: `/v3/projects/4eacc288ff1c40c8877635b387ffc3bc` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:37.722795 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0485c8f-606e-4d86-a080-28e86886a277 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4eacc288ff1c40c8877635b387ffc3bc)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:37.725865 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0485c8f-606e-4d86-a080-28e86886a277 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:37.804637 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d0485c8f-606e-4d86-a080-28e86886a277 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4eacc288ff1c40c8877635b387ffc3bc', 'request_id': 'req-d0485c8f-606e-4d86-a080-28e86886a277'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:37.815046 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d0485c8f-606e-4d86-a080-28e86886a277 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4eacc288ff1c40c8877635b387ffc3bc', 'request_id': 'req-d0485c8f-606e-4d86-a080-28e86886a277'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:37.822403 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d0485c8f-606e-4d86-a080-28e86886a277 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4eacc288ff1c40c8877635b387ffc3bc', 'request_id': 'req-d0485c8f-606e-4d86-a080-28e86886a277'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:37.835775 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d0485c8f-606e-4d86-a080-28e86886a277 admin admin] The token cache is being invalidate because project 4eacc288ff1c40c8877635b387ffc3bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:48:37.836085 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d0485c8f-606e-4d86-a080-28e86886a277 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:37.836864 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d0485c8f-606e-4d86-a080-28e86886a277 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:37.841937 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1268/2548] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:48:37 2022] DELETE /identity/v3/projects/4eacc288ff1c40c8877635b387ffc3bc => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:37.844658 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd95d8d-feb2-4be1-a186-4a06f4797e41 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:37.845025 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd95d8d-feb2-4be1-a186-4a06f4797e41 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:37.845332 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd95d8d-feb2-4be1-a186-4a06f4797e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:37.882280 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-7bd95d8d-feb2-4be1-a186-4a06f4797e41 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:37.883868 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7bd95d8d-feb2-4be1-a186-4a06f4797e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:37.971383 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1281/2549] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:37.979496 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb552f5e-48b6-4b87-be97-4c4a69591c12 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:37.980107 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb552f5e-48b6-4b87-be97-4c4a69591c12 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:37.980669 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb552f5e-48b6-4b87-be97-4c4a69591c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:38.024816 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-bb552f5e-48b6-4b87-be97-4c4a69591c12 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:38.026133 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-bb552f5e-48b6-4b87-be97-4c4a69591c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:38.039421 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1269/2550] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:38.044158 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd95d8d-feb2-4be1-a186-4a06f4797e41 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:38.058151 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd95d8d-feb2-4be1-a186-4a06f4797e41 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:38.064476 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75895d24-73ba-4953-8d7b-17784604ab33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:38.065114 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75895d24-73ba-4953-8d7b-17784604ab33 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:38.065400 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75895d24-73ba-4953-8d7b-17784604ab33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:38.065676 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75895d24-73ba-4953-8d7b-17784604ab33 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:38.067194 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75895d24-73ba-4953-8d7b-17784604ab33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:48:38.067508 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75895d24-73ba-4953-8d7b-17784604ab33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:38.069148 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75895d24-73ba-4953-8d7b-17784604ab33 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:38.083565 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1282/2551] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:48:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:48:38.087854 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb552f5e-48b6-4b87-be97-4c4a69591c12 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:38.103489 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb552f5e-48b6-4b87-be97-4c4a69591c12 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:38.109218 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d61cdb-c844-4530-be4d-60a0ea8171bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:38.110048 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d61cdb-c844-4530-be4d-60a0ea8171bb admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:38.110541 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d61cdb-c844-4530-be4d-60a0ea8171bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:38.111014 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d61cdb-c844-4530-be4d-60a0ea8171bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:38.111817 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d61cdb-c844-4530-be4d-60a0ea8171bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:38.114357 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d61cdb-c844-4530-be4d-60a0ea8171bb admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:38.149265 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1270/2552] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:48:38 2022] POST /identity/v3/projects => generated 385 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:48:38.154092 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75895d24-73ba-4953-8d7b-17784604ab33 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:38.166615 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75895d24-73ba-4953-8d7b-17784604ab33 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:38.230129 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c297c859-eb9b-42e6-accf-aa7ae2bde9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:38.230754 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c297c859-eb9b-42e6-accf-aa7ae2bde9ed admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:38.231352 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c297c859-eb9b-42e6-accf-aa7ae2bde9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:38.231624 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c297c859-eb9b-42e6-accf-aa7ae2bde9ed admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:38.232254 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c297c859-eb9b-42e6-accf-aa7ae2bde9ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:38.234384 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c297c859-eb9b-42e6-accf-aa7ae2bde9ed admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:38.264871 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1283/2553] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:48:38 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:48:38.266106 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d61cdb-c844-4530-be4d-60a0ea8171bb admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:38.281297 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d61cdb-c844-4530-be4d-60a0ea8171bb admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:38.290687 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ab05aa-ae9b-466b-8851-be787ca5211e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:38.291556 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab05aa-ae9b-466b-8851-be787ca5211e admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:38.292003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab05aa-ae9b-466b-8851-be787ca5211e admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:38.292487 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab05aa-ae9b-466b-8851-be787ca5211e admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:38.293236 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ab05aa-ae9b-466b-8851-be787ca5211e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:38.295083 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ab05aa-ae9b-466b-8851-be787ca5211e admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:38.311225 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1271/2554] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:48:38 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:48:38.318941 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c297c859-eb9b-42e6-accf-aa7ae2bde9ed admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:38.337565 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c297c859-eb9b-42e6-accf-aa7ae2bde9ed admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:38.342987 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ffe653-d532-4d32-9141-ae4cd51beaa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:38.343676 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ffe653-d532-4d32-9141-ae4cd51beaa4 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:38.345516 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ffe653-d532-4d32-9141-ae4cd51beaa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:38.345516 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ffe653-d532-4d32-9141-ae4cd51beaa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:38.345516 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ffe653-d532-4d32-9141-ae4cd51beaa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:38.348353 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ffe653-d532-4d32-9141-ae4cd51beaa4 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:38.356305 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:38.368453 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1284/2555] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:48:38 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:48:38.369564 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ab05aa-ae9b-466b-8851-be787ca5211e admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:38.379963 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ab05aa-ae9b-466b-8851-be787ca5211e admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:38.396823 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99164eb0-64c0-4244-8246-5ebf885974f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:38.397558 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99164eb0-64c0-4244-8246-5ebf885974f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:38.397835 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99164eb0-64c0-4244-8246-5ebf885974f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:38.398136 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99164eb0-64c0-4244-8246-5ebf885974f1 admin admin] PATH_INFO: `/v3/projects/10a3c501ae9a4ea5a317c580568a0210/users/7986c65b8b494ff581c9526ff6cea016/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:38.422279 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99164eb0-64c0-4244-8246-5ebf885974f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10a3c501ae9a4ea5a317c580568a0210, user_id=7986c65b8b494ff581c9526ff6cea016, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:38.430580 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99164eb0-64c0-4244-8246-5ebf885974f1 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:38.468261 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1272/2556] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:48:38 2022] PUT /identity/v3/projects/10a3c501ae9a4ea5a317c580568a0210/users/7986c65b8b494ff581c9526ff6cea016/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:38.479328 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca702d02-626e-408e-a1de-b28e997d0b9a None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:38.479618 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca702d02-626e-408e-a1de-b28e997d0b9a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:38.479916 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca702d02-626e-408e-a1de-b28e997d0b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:38.533107 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-ca702d02-626e-408e-a1de-b28e997d0b9a None None] MFA Rules not processed for user `7986c65b8b494ff581c9526ff6cea016`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:38.535451 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ca702d02-626e-408e-a1de-b28e997d0b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:38.647071 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f30d3-0016-4af8-9745-af8b8fbd54e5 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:38.647871 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1285/2557] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:48:38.648091 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f30d3-0016-4af8-9745-af8b8fbd54e5 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:38.648373 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f30d3-0016-4af8-9745-af8b8fbd54e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:38.660498 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca702d02-626e-408e-a1de-b28e997d0b9a None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:38.662614 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ca702d02-626e-408e-a1de-b28e997d0b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:38.672157 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-917f30d3-0016-4af8-9745-af8b8fbd54e5 None None] MFA Rules not processed for user `55a6ab5102034cc68a60218d0d434f25`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:38.673591 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-917f30d3-0016-4af8-9745-af8b8fbd54e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:38.737142 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1273/2558] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:38.751232 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca702d02-626e-408e-a1de-b28e997d0b9a None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:38.758861 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb5e56-4598-486a-948c-5ee9611394c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf14b64bf04b2ba74d2fe1fd41ff71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:38.759068 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb5e56-4598-486a-948c-5ee9611394c6 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:38.759386 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb5e56-4598-486a-948c-5ee9611394c6 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:38.759692 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb5e56-4598-486a-948c-5ee9611394c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:38.769556 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccb5e56-4598-486a-948c-5ee9611394c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:38.771398 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccb5e56-4598-486a-948c-5ee9611394c6 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:38.782693 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1286/2559] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:48:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:48:46.274686 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f30d3-0016-4af8-9745-af8b8fbd54e5 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:46.277997 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:46.338804 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f30d3-0016-4af8-9745-af8b8fbd54e5 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:46.343665 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640eddfa-fada-4cc9-8836-7d9817fdb4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:46.344306 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640eddfa-fada-4cc9-8836-7d9817fdb4af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:46.344577 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640eddfa-fada-4cc9-8836-7d9817fdb4af admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:46.344843 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640eddfa-fada-4cc9-8836-7d9817fdb4af admin admin] PATH_INFO: `/v3/users/7986c65b8b494ff581c9526ff6cea016` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:46.346060 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640eddfa-fada-4cc9-8836-7d9817fdb4af admin admin] RBAC: Authorizing `identity:delete_user(user_id=7986c65b8b494ff581c9526ff6cea016)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:46.347973 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640eddfa-fada-4cc9-8836-7d9817fdb4af admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:46.394577 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-640eddfa-fada-4cc9-8836-7d9817fdb4af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7986c65b8b494ff581c9526ff6cea016'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:46.401980 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-640eddfa-fada-4cc9-8836-7d9817fdb4af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7986c65b8b494ff581c9526ff6cea016'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:46.402662 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-640eddfa-fada-4cc9-8836-7d9817fdb4af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7986c65b8b494ff581c9526ff6cea016'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:46.410042 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-640eddfa-fada-4cc9-8836-7d9817fdb4af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7986c65b8b494ff581c9526ff6cea016'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:46.410807 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-640eddfa-fada-4cc9-8836-7d9817fdb4af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7986c65b8b494ff581c9526ff6cea016'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:46.428661 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1274/2560] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:48:46 2022] DELETE /identity/v3/users/7986c65b8b494ff581c9526ff6cea016 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:46.433117 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5c9fb-f4a9-450b-ad3b-ff65e4e1ef18 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:46.433602 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5c9fb-f4a9-450b-ad3b-ff65e4e1ef18 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:46.433713 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5c9fb-f4a9-450b-ad3b-ff65e4e1ef18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:46.452627 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-73f5c9fb-f4a9-450b-ad3b-ff65e4e1ef18 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:46.453798 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-73f5c9fb-f4a9-450b-ad3b-ff65e4e1ef18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:46.515376 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1287/2561] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:46.525649 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640eddfa-fada-4cc9-8836-7d9817fdb4af admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:46.527059 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-640eddfa-fada-4cc9-8836-7d9817fdb4af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:46.586877 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640eddfa-fada-4cc9-8836-7d9817fdb4af admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:46.591274 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ce301-0aff-47c4-ae76-7e2338a31cd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:46.591906 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ce301-0aff-47c4-ae76-7e2338a31cd9 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:46.592367 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ce301-0aff-47c4-ae76-7e2338a31cd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:46.592641 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ce301-0aff-47c4-ae76-7e2338a31cd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:46.602360 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5ce301-0aff-47c4-ae76-7e2338a31cd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:46.603859 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5ce301-0aff-47c4-ae76-7e2338a31cd9 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:46.613760 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1275/2562] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:48:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:48:46.899346 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5c9fb-f4a9-450b-ad3b-ff65e4e1ef18 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:46.900884 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-73f5c9fb-f4a9-450b-ad3b-ff65e4e1ef18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:46.914067 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5c9fb-f4a9-450b-ad3b-ff65e4e1ef18 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:46.918579 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaeb7ee-20d6-4c37-95a7-4850516e12a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:46.919217 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaeb7ee-20d6-4c37-95a7-4850516e12a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:46.919498 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaeb7ee-20d6-4c37-95a7-4850516e12a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:46.919773 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaeb7ee-20d6-4c37-95a7-4850516e12a3 admin admin] PATH_INFO: `/v3/projects/10a3c501ae9a4ea5a317c580568a0210` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:46.920886 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaeb7ee-20d6-4c37-95a7-4850516e12a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10a3c501ae9a4ea5a317c580568a0210)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:46.922999 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaeb7ee-20d6-4c37-95a7-4850516e12a3 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:46.935473 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:46.973105 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-fcaeb7ee-20d6-4c37-95a7-4850516e12a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10a3c501ae9a4ea5a317c580568a0210', 'request_id': 'req-fcaeb7ee-20d6-4c37-95a7-4850516e12a3'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:46.980276 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-fcaeb7ee-20d6-4c37-95a7-4850516e12a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10a3c501ae9a4ea5a317c580568a0210', 'request_id': 'req-fcaeb7ee-20d6-4c37-95a7-4850516e12a3'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:46.987875 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-fcaeb7ee-20d6-4c37-95a7-4850516e12a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10a3c501ae9a4ea5a317c580568a0210', 'request_id': 'req-fcaeb7ee-20d6-4c37-95a7-4850516e12a3'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:46.999038 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-fcaeb7ee-20d6-4c37-95a7-4850516e12a3 admin admin] The token cache is being invalidate because project 10a3c501ae9a4ea5a317c580568a0210 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:48:46.999336 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-fcaeb7ee-20d6-4c37-95a7-4850516e12a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:46.999962 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-fcaeb7ee-20d6-4c37-95a7-4850516e12a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:47.001447 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1288/2563] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:48:46 2022] DELETE /identity/v3/projects/10a3c501ae9a4ea5a317c580568a0210 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:47.007739 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ccb83-e3ff-4f4f-9884-96fbc57459da None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:47.008073 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ccb83-e3ff-4f4f-9884-96fbc57459da None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:47.008544 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ccb83-e3ff-4f4f-9884-96fbc57459da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:47.028588 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-d09ccb83-e3ff-4f4f-9884-96fbc57459da None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:47.029773 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d09ccb83-e3ff-4f4f-9884-96fbc57459da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:47.049810 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:47.100081 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1276/2564] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:47.109659 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1ef11-b974-410e-a34f-41a0d11ce92b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:47.109659 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1ef11-b974-410e-a34f-41a0d11ce92b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:47.109875 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1ef11-b974-410e-a34f-41a0d11ce92b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:47.130528 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d8d1ef11-b974-410e-a34f-41a0d11ce92b None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:47.131818 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d8d1ef11-b974-410e-a34f-41a0d11ce92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:47.140298 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1289/2565] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:47.144455 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ccb83-e3ff-4f4f-9884-96fbc57459da None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:47.154142 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ccb83-e3ff-4f4f-9884-96fbc57459da None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:47.159017 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4589de07-dc80-45ba-a32b-42cab5ac5af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:47.159634 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4589de07-dc80-45ba-a32b-42cab5ac5af9 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:47.159912 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4589de07-dc80-45ba-a32b-42cab5ac5af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:47.160183 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4589de07-dc80-45ba-a32b-42cab5ac5af9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:47.160830 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4589de07-dc80-45ba-a32b-42cab5ac5af9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:48:47.161623 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4589de07-dc80-45ba-a32b-42cab5ac5af9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:47.163264 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4589de07-dc80-45ba-a32b-42cab5ac5af9 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:47.175223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1277/2566] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:48:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:48:47.180305 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d1ef11-b974-410e-a34f-41a0d11ce92b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:47.190008 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d1ef11-b974-410e-a34f-41a0d11ce92b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:47.196318 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb51fcc-b427-4e05-8db6-9edbd070bb3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:47.197142 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb51fcc-b427-4e05-8db6-9edbd070bb3a admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:47.197683 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb51fcc-b427-4e05-8db6-9edbd070bb3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:47.198167 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb51fcc-b427-4e05-8db6-9edbd070bb3a admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:47.198808 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb51fcc-b427-4e05-8db6-9edbd070bb3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:47.200901 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb51fcc-b427-4e05-8db6-9edbd070bb3a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:47.230481 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1290/2567] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:48:47 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:48:47.235079 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4589de07-dc80-45ba-a32b-42cab5ac5af9 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:47.247961 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4589de07-dc80-45ba-a32b-42cab5ac5af9 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:47.319728 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a37bf7-8670-4968-9b8f-eac021b66356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:47.320611 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a37bf7-8670-4968-9b8f-eac021b66356 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:47.323908 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a37bf7-8670-4968-9b8f-eac021b66356 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:47.324243 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a37bf7-8670-4968-9b8f-eac021b66356 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:47.324917 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a37bf7-8670-4968-9b8f-eac021b66356 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:47.328058 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a37bf7-8670-4968-9b8f-eac021b66356 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:47.355907 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1278/2568] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:48:47 2022] POST /identity/v3/users => generated 293 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:48:47.360173 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb51fcc-b427-4e05-8db6-9edbd070bb3a admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:47.372769 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb51fcc-b427-4e05-8db6-9edbd070bb3a admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:47.378404 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88643ea-cac1-4156-b1a0-926ea3d40d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:47.379062 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88643ea-cac1-4156-b1a0-926ea3d40d7a admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:47.379344 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88643ea-cac1-4156-b1a0-926ea3d40d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:47.379617 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88643ea-cac1-4156-b1a0-926ea3d40d7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:47.380396 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88643ea-cac1-4156-b1a0-926ea3d40d7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:47.381903 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88643ea-cac1-4156-b1a0-926ea3d40d7a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:47.397235 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1291/2569] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:48:47.399178 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a37bf7-8670-4968-9b8f-eac021b66356 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:47.416435 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a37bf7-8670-4968-9b8f-eac021b66356 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:47.421270 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46205ddf-aa38-432e-aebd-08686e5bba02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:47.421890 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46205ddf-aa38-432e-aebd-08686e5bba02 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:47.422172 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46205ddf-aa38-432e-aebd-08686e5bba02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:47.422444 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46205ddf-aa38-432e-aebd-08686e5bba02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:47.423078 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46205ddf-aa38-432e-aebd-08686e5bba02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:47.424777 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46205ddf-aa38-432e-aebd-08686e5bba02 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:47.436575 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1279/2570] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:48:47.441037 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88643ea-cac1-4156-b1a0-926ea3d40d7a admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:47.451553 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88643ea-cac1-4156-b1a0-926ea3d40d7a admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:47.456193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a641290-8bcc-4329-967f-19c679afc9c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:47.456896 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a641290-8bcc-4329-967f-19c679afc9c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:47.457385 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a641290-8bcc-4329-967f-19c679afc9c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:47.457529 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a641290-8bcc-4329-967f-19c679afc9c2 admin admin] PATH_INFO: `/v3/projects/aca001753e754b9f806365b5dbbbd61d/users/afe4fd6e215d4395879ef6d134ba8ae8/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:47.472275 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a641290-8bcc-4329-967f-19c679afc9c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aca001753e754b9f806365b5dbbbd61d, user_id=afe4fd6e215d4395879ef6d134ba8ae8, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:47.475738 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a641290-8bcc-4329-967f-19c679afc9c2 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:47.491228 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1292/2571] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:48:47 2022] PUT /identity/v3/projects/aca001753e754b9f806365b5dbbbd61d/users/afe4fd6e215d4395879ef6d134ba8ae8/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:47.501645 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85937ca0-f209-4253-b2a3-9e4d6673aeca None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:47.501935 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85937ca0-f209-4253-b2a3-9e4d6673aeca None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:47.502217 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85937ca0-f209-4253-b2a3-9e4d6673aeca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:47.529898 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-85937ca0-f209-4253-b2a3-9e4d6673aeca None None] MFA Rules not processed for user `afe4fd6e215d4395879ef6d134ba8ae8`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:47.531085 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-85937ca0-f209-4253-b2a3-9e4d6673aeca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:47.596444 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1280/2572] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:48:47.601635 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a641290-8bcc-4329-967f-19c679afc9c2 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:47.666445 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a641290-8bcc-4329-967f-19c679afc9c2 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:47.671392 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdc3e26-cc7f-49cb-b9bc-73d479a9de07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:47.672021 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc3e26-cc7f-49cb-b9bc-73d479a9de07 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:47.672408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc3e26-cc7f-49cb-b9bc-73d479a9de07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:47.672807 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc3e26-cc7f-49cb-b9bc-73d479a9de07 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:47.673630 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdc3e26-cc7f-49cb-b9bc-73d479a9de07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:47.675881 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdc3e26-cc7f-49cb-b9bc-73d479a9de07 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:47.706280 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1293/2573] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:48:47 2022] POST /identity/v3/projects => generated 387 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:48:47.707662 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85937ca0-f209-4253-b2a3-9e4d6673aeca None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:47.719260 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85937ca0-f209-4253-b2a3-9e4d6673aeca None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:47.738407 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:47.785525 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b2b42c-718c-4214-81c9-38c233569314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:47.786714 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b2b42c-718c-4214-81c9-38c233569314 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:47.786993 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b2b42c-718c-4214-81c9-38c233569314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:47.787260 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b2b42c-718c-4214-81c9-38c233569314 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:47.787900 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b2b42c-718c-4214-81c9-38c233569314 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:47.789802 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b2b42c-718c-4214-81c9-38c233569314 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:47.814619 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1281/2574] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:48:47 2022] POST /identity/v3/users => generated 300 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:48:47.816672 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdc3e26-cc7f-49cb-b9bc-73d479a9de07 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:47.827761 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdc3e26-cc7f-49cb-b9bc-73d479a9de07 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:47.832332 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c4e2b-8bba-4d0d-ad64-99174abc497c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:47.832996 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c4e2b-8bba-4d0d-ad64-99174abc497c admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:47.833297 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c4e2b-8bba-4d0d-ad64-99174abc497c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:47.833566 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c4e2b-8bba-4d0d-ad64-99174abc497c admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:47.834224 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c4e2b-8bba-4d0d-ad64-99174abc497c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:47.835807 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c4e2b-8bba-4d0d-ad64-99174abc497c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:47.846492 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1294/2575] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:48:47.851171 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b2b42c-718c-4214-81c9-38c233569314 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:47.862784 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b2b42c-718c-4214-81c9-38c233569314 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:47.868577 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9216fc2-3c80-4d9b-ba49-5a3b752aeff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:47.869233 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9216fc2-3c80-4d9b-ba49-5a3b752aeff2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:47.869509 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9216fc2-3c80-4d9b-ba49-5a3b752aeff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:47.869777 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9216fc2-3c80-4d9b-ba49-5a3b752aeff2 admin admin] PATH_INFO: `/v3/projects/e9f8f66b8c0845d6823f9688ccca032c/users/e54b8f0c40384cee9f8db86d97680ced/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:47.884726 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9216fc2-3c80-4d9b-ba49-5a3b752aeff2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9f8f66b8c0845d6823f9688ccca032c, user_id=e54b8f0c40384cee9f8db86d97680ced, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:47.887543 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9216fc2-3c80-4d9b-ba49-5a3b752aeff2 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:47.900963 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1282/2576] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:48:47 2022] PUT /identity/v3/projects/e9f8f66b8c0845d6823f9688ccca032c/users/e54b8f0c40384cee9f8db86d97680ced/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:47.905027 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c4e2b-8bba-4d0d-ad64-99174abc497c admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:47.915599 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c4e2b-8bba-4d0d-ad64-99174abc497c admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:47.920361 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:47.976998 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9abbe3-3578-45a2-9f47-404b7bdc77e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:47.977506 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9abbe3-3578-45a2-9f47-404b7bdc77e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:47.977800 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9abbe3-3578-45a2-9f47-404b7bdc77e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:47.978116 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9abbe3-3578-45a2-9f47-404b7bdc77e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:47.978748 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9abbe3-3578-45a2-9f47-404b7bdc77e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:47.980337 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9abbe3-3578-45a2-9f47-404b7bdc77e0 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:47.990882 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1295/2577] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:48:47.995411 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9216fc2-3c80-4d9b-ba49-5a3b752aeff2 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:48.008976 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9216fc2-3c80-4d9b-ba49-5a3b752aeff2 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:48.013618 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fb517e-f39b-4f7a-b83a-e6af2831e0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:48.014291 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb517e-f39b-4f7a-b83a-e6af2831e0f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:48.014567 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb517e-f39b-4f7a-b83a-e6af2831e0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:48.014836 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb517e-f39b-4f7a-b83a-e6af2831e0f4 admin admin] PATH_INFO: `/v3/projects/e9f8f66b8c0845d6823f9688ccca032c/users/e54b8f0c40384cee9f8db86d97680ced/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:48.016735 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fb517e-f39b-4f7a-b83a-e6af2831e0f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9f8f66b8c0845d6823f9688ccca032c, user_id=e54b8f0c40384cee9f8db86d97680ced, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:48.019374 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fb517e-f39b-4f7a-b83a-e6af2831e0f4 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:48.029267 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1283/2578] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:48:47 2022] PUT /identity/v3/projects/e9f8f66b8c0845d6823f9688ccca032c/users/e54b8f0c40384cee9f8db86d97680ced/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:48:48.040324 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d3629-1150-4df5-a672-80006492f7fc None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:48.040636 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d3629-1150-4df5-a672-80006492f7fc None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:48.040911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d3629-1150-4df5-a672-80006492f7fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:48.070531 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-e26d3629-1150-4df5-a672-80006492f7fc None None] MFA Rules not processed for user `e54b8f0c40384cee9f8db86d97680ced`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:48.071737 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e26d3629-1150-4df5-a672-80006492f7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:48.141534 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1296/2579] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:48 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:48.154210 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fb517e-f39b-4f7a-b83a-e6af2831e0f4 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:48.155892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-87fb517e-f39b-4f7a-b83a-e6af2831e0f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:48.215773 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fb517e-f39b-4f7a-b83a-e6af2831e0f4 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:48.219316 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9b7645-d163-46c6-a806-58900147eedd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:48.219908 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b7645-d163-46c6-a806-58900147eedd service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:48.220179 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b7645-d163-46c6-a806-58900147eedd service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:48.220442 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b7645-d163-46c6-a806-58900147eedd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:48.230524 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9b7645-d163-46c6-a806-58900147eedd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:48.232055 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9b7645-d163-46c6-a806-58900147eedd service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:48.242368 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1284/2580] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:48:48 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:48:48.311316 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d3629-1150-4df5-a672-80006492f7fc None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:48.376783 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26d3629-1150-4df5-a672-80006492f7fc None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:48.382119 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920e8464-4041-4c74-bde3-bf46f8f38a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:48.382787 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e8464-4041-4c74-bde3-bf46f8f38a62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:48.383064 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e8464-4041-4c74-bde3-bf46f8f38a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:48.383360 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e8464-4041-4c74-bde3-bf46f8f38a62 admin admin] PATH_INFO: `/v3/users/afe4fd6e215d4395879ef6d134ba8ae8` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:48.384495 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920e8464-4041-4c74-bde3-bf46f8f38a62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afe4fd6e215d4395879ef6d134ba8ae8)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:48.386528 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920e8464-4041-4c74-bde3-bf46f8f38a62 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:48.424292 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:48.440569 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-920e8464-4041-4c74-bde3-bf46f8f38a62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afe4fd6e215d4395879ef6d134ba8ae8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:48.441234 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-920e8464-4041-4c74-bde3-bf46f8f38a62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afe4fd6e215d4395879ef6d134ba8ae8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:48.454051 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-920e8464-4041-4c74-bde3-bf46f8f38a62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afe4fd6e215d4395879ef6d134ba8ae8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:48.463924 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-920e8464-4041-4c74-bde3-bf46f8f38a62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afe4fd6e215d4395879ef6d134ba8ae8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:48.471014 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-920e8464-4041-4c74-bde3-bf46f8f38a62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afe4fd6e215d4395879ef6d134ba8ae8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:48.476948 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1297/2581] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:48:48 2022] DELETE /identity/v3/users/afe4fd6e215d4395879ef6d134ba8ae8 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:48.481393 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15631f6-b710-470a-b7e5-8127e344a812 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:48.482262 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15631f6-b710-470a-b7e5-8127e344a812 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:48.482728 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15631f6-b710-470a-b7e5-8127e344a812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:48.504323 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-a15631f6-b710-470a-b7e5-8127e344a812 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:48.506167 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a15631f6-b710-470a-b7e5-8127e344a812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:48.577602 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1285/2582] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:48.588062 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920e8464-4041-4c74-bde3-bf46f8f38a62 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:48.589697 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-920e8464-4041-4c74-bde3-bf46f8f38a62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:48.656255 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920e8464-4041-4c74-bde3-bf46f8f38a62 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:48.661192 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7571b6-142c-474a-9548-7919c51cb9ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:48.661948 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7571b6-142c-474a-9548-7919c51cb9ca service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:48.662341 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7571b6-142c-474a-9548-7919c51cb9ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:48.662636 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7571b6-142c-474a-9548-7919c51cb9ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:48.673297 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7571b6-142c-474a-9548-7919c51cb9ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:48.674758 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7571b6-142c-474a-9548-7919c51cb9ca service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:48.684758 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1298/2583] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:48:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:48:48.990907 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15631f6-b710-470a-b7e5-8127e344a812 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:48.992355 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a15631f6-b710-470a-b7e5-8127e344a812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:49.010306 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15631f6-b710-470a-b7e5-8127e344a812 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:49.015054 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d942317-1655-4149-ad8f-49b7adc58fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:49.015890 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d942317-1655-4149-ad8f-49b7adc58fd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:49.015890 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d942317-1655-4149-ad8f-49b7adc58fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:49.016232 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d942317-1655-4149-ad8f-49b7adc58fd0 admin admin] PATH_INFO: `/v3/projects/aca001753e754b9f806365b5dbbbd61d` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:49.017351 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d942317-1655-4149-ad8f-49b7adc58fd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aca001753e754b9f806365b5dbbbd61d)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:49.019270 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d942317-1655-4149-ad8f-49b7adc58fd0 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:49.037988 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:49.075268 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-2d942317-1655-4149-ad8f-49b7adc58fd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aca001753e754b9f806365b5dbbbd61d', 'request_id': 'req-2d942317-1655-4149-ad8f-49b7adc58fd0'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:49.083595 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-2d942317-1655-4149-ad8f-49b7adc58fd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aca001753e754b9f806365b5dbbbd61d', 'request_id': 'req-2d942317-1655-4149-ad8f-49b7adc58fd0'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:49.093515 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-2d942317-1655-4149-ad8f-49b7adc58fd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aca001753e754b9f806365b5dbbbd61d', 'request_id': 'req-2d942317-1655-4149-ad8f-49b7adc58fd0'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:49.106242 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-2d942317-1655-4149-ad8f-49b7adc58fd0 admin admin] The token cache is being invalidate because project aca001753e754b9f806365b5dbbbd61d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:48:49.106366 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-2d942317-1655-4149-ad8f-49b7adc58fd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:49.106966 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-2d942317-1655-4149-ad8f-49b7adc58fd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:49.108551 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1286/2584] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:48:48 2022] DELETE /identity/v3/projects/aca001753e754b9f806365b5dbbbd61d => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:49.112411 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7571b6-142c-474a-9548-7919c51cb9ca service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:49.122625 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7571b6-142c-474a-9548-7919c51cb9ca service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:49.180870 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1477f4a4-c4a4-4e44-ad13-f180110d2e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:49.181491 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477f4a4-c4a4-4e44-ad13-f180110d2e70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:49.181768 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477f4a4-c4a4-4e44-ad13-f180110d2e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:49.182112 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477f4a4-c4a4-4e44-ad13-f180110d2e70 admin admin] PATH_INFO: `/v3/users/e54b8f0c40384cee9f8db86d97680ced` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:49.183172 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1477f4a4-c4a4-4e44-ad13-f180110d2e70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e54b8f0c40384cee9f8db86d97680ced)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:49.185089 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1477f4a4-c4a4-4e44-ad13-f180110d2e70 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:49.245544 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1477f4a4-c4a4-4e44-ad13-f180110d2e70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e54b8f0c40384cee9f8db86d97680ced'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:49.246384 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1477f4a4-c4a4-4e44-ad13-f180110d2e70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e54b8f0c40384cee9f8db86d97680ced'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:49.252124 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:49.261922 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1477f4a4-c4a4-4e44-ad13-f180110d2e70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e54b8f0c40384cee9f8db86d97680ced'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:49.269601 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1477f4a4-c4a4-4e44-ad13-f180110d2e70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e54b8f0c40384cee9f8db86d97680ced'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:49.280008 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-1477f4a4-c4a4-4e44-ad13-f180110d2e70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e54b8f0c40384cee9f8db86d97680ced'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:49.285839 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1299/2585] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:48:49 2022] DELETE /identity/v3/users/e54b8f0c40384cee9f8db86d97680ced => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:48:49.573487 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d942317-1655-4149-ad8f-49b7adc58fd0 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:49.575008 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-2d942317-1655-4149-ad8f-49b7adc58fd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:49.589295 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d942317-1655-4149-ad8f-49b7adc58fd0 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:49.651815 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ed6e90-3b35-4776-b621-221ada581978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:49.652454 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed6e90-3b35-4776-b621-221ada581978 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:49.652732 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed6e90-3b35-4776-b621-221ada581978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:49.653001 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed6e90-3b35-4776-b621-221ada581978 admin admin] PATH_INFO: `/v3/projects/e9f8f66b8c0845d6823f9688ccca032c` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:49.654148 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ed6e90-3b35-4776-b621-221ada581978 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9f8f66b8c0845d6823f9688ccca032c)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:49.656044 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ed6e90-3b35-4776-b621-221ada581978 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:49.704336 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-36ed6e90-3b35-4776-b621-221ada581978 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9f8f66b8c0845d6823f9688ccca032c', 'request_id': 'req-36ed6e90-3b35-4776-b621-221ada581978'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:49.714573 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-36ed6e90-3b35-4776-b621-221ada581978 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9f8f66b8c0845d6823f9688ccca032c', 'request_id': 'req-36ed6e90-3b35-4776-b621-221ada581978'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:49.722157 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-36ed6e90-3b35-4776-b621-221ada581978 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9f8f66b8c0845d6823f9688ccca032c', 'request_id': 'req-36ed6e90-3b35-4776-b621-221ada581978'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:49.733104 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-36ed6e90-3b35-4776-b621-221ada581978 admin admin] The token cache is being invalidate because project e9f8f66b8c0845d6823f9688ccca032c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:48:49.733361 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-36ed6e90-3b35-4776-b621-221ada581978 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:49.734270 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-36ed6e90-3b35-4776-b621-221ada581978 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:49.736007 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1287/2586] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:48:49 2022] DELETE /identity/v3/projects/e9f8f66b8c0845d6823f9688ccca032c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:49.741663 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cef215-088d-4443-83df-09d0abf55e0e None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:49.742230 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cef215-088d-4443-83df-09d0abf55e0e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:49.742230 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cef215-088d-4443-83df-09d0abf55e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:49.763986 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-a0cef215-088d-4443-83df-09d0abf55e0e None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:49.765096 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a0cef215-088d-4443-83df-09d0abf55e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:49.831295 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1300/2587] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:49.839501 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6decbd9-475e-4a59-bcf6-d7b8b32c5577 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:49.839787 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6decbd9-475e-4a59-bcf6-d7b8b32c5577 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:49.840064 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6decbd9-475e-4a59-bcf6-d7b8b32c5577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:49.865809 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-f6decbd9-475e-4a59-bcf6-d7b8b32c5577 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:49.867103 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-f6decbd9-475e-4a59-bcf6-d7b8b32c5577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:49.876205 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1288/2588] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:49.881132 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cef215-088d-4443-83df-09d0abf55e0e None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:49.890941 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cef215-088d-4443-83df-09d0abf55e0e None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:49.896589 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d792df20-70b6-404d-b454-341cacd14ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:49.897202 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792df20-70b6-404d-b454-341cacd14ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:49.897479 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792df20-70b6-404d-b454-341cacd14ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:49.897747 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792df20-70b6-404d-b454-341cacd14ce8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:49.898433 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d792df20-70b6-404d-b454-341cacd14ce8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:48:49.898733 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d792df20-70b6-404d-b454-341cacd14ce8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:49.907405 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d792df20-70b6-404d-b454-341cacd14ce8 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:49.913304 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1301/2589] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:48:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:48:49.917552 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6decbd9-475e-4a59-bcf6-d7b8b32c5577 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:49.927715 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6decbd9-475e-4a59-bcf6-d7b8b32c5577 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:49.932821 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4791a957-2bd8-4417-a3f0-ba57630b8984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:49.933459 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791a957-2bd8-4417-a3f0-ba57630b8984 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:49.933731 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791a957-2bd8-4417-a3f0-ba57630b8984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:49.934098 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791a957-2bd8-4417-a3f0-ba57630b8984 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:49.934737 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4791a957-2bd8-4417-a3f0-ba57630b8984 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:49.936634 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4791a957-2bd8-4417-a3f0-ba57630b8984 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:49.968296 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1289/2590] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:48:49 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:48:49.972529 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d792df20-70b6-404d-b454-341cacd14ce8 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:49.983082 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d792df20-70b6-404d-b454-341cacd14ce8 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:50.040568 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9e9098-1dba-4b7f-9676-0760f1b1eb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:50.041099 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e9098-1dba-4b7f-9676-0760f1b1eb10 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:50.041380 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e9098-1dba-4b7f-9676-0760f1b1eb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:50.041656 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e9098-1dba-4b7f-9676-0760f1b1eb10 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:50.042334 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9e9098-1dba-4b7f-9676-0760f1b1eb10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:50.044277 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9e9098-1dba-4b7f-9676-0760f1b1eb10 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:50.065914 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1302/2591] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:48:49 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:48:50.070480 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4791a957-2bd8-4417-a3f0-ba57630b8984 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:50.076921 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:50.084638 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4791a957-2bd8-4417-a3f0-ba57630b8984 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:50.091484 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f2550-18c3-4e4a-b8cb-489a6b269502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:50.092097 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f2550-18c3-4e4a-b8cb-489a6b269502 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:50.092664 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f2550-18c3-4e4a-b8cb-489a6b269502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:50.093009 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f2550-18c3-4e4a-b8cb-489a6b269502 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:50.093652 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685f2550-18c3-4e4a-b8cb-489a6b269502 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:50.095315 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685f2550-18c3-4e4a-b8cb-489a6b269502 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:50.109706 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1290/2592] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:48:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:48:50.110660 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9e9098-1dba-4b7f-9676-0760f1b1eb10 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:50.121676 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9e9098-1dba-4b7f-9676-0760f1b1eb10 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:50.126953 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a0b782-3b6c-4f72-bef5-e5b4a5df0495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:50.127666 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0b782-3b6c-4f72-bef5-e5b4a5df0495 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:50.128182 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0b782-3b6c-4f72-bef5-e5b4a5df0495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:50.128586 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0b782-3b6c-4f72-bef5-e5b4a5df0495 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:50.129379 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a0b782-3b6c-4f72-bef5-e5b4a5df0495 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:50.131807 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a0b782-3b6c-4f72-bef5-e5b4a5df0495 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:50.146645 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1303/2593] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:48:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:48:50.148687 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f2550-18c3-4e4a-b8cb-489a6b269502 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:50.160967 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f2550-18c3-4e4a-b8cb-489a6b269502 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:50.165639 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09015ccd-ade0-4d00-ae99-178083eddd04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:50.166348 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09015ccd-ade0-4d00-ae99-178083eddd04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:50.166627 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09015ccd-ade0-4d00-ae99-178083eddd04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:50.166985 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09015ccd-ade0-4d00-ae99-178083eddd04 admin admin] PATH_INFO: `/v3/projects/c0bb76f8f11445ad884b64dc0fdde8d6/users/113c637cb4b543b19e72700a4be5f7be/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:50.189236 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09015ccd-ade0-4d00-ae99-178083eddd04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0bb76f8f11445ad884b64dc0fdde8d6, user_id=113c637cb4b543b19e72700a4be5f7be, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:50.193943 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09015ccd-ade0-4d00-ae99-178083eddd04 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:50.211191 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1291/2594] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:48:50 2022] PUT /identity/v3/projects/c0bb76f8f11445ad884b64dc0fdde8d6/users/113c637cb4b543b19e72700a4be5f7be/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:50.222536 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0029b57-ef5a-4190-8f66-fe6799580f41 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:50.223129 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0029b57-ef5a-4190-8f66-fe6799580f41 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:50.223617 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0029b57-ef5a-4190-8f66-fe6799580f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:50.264541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-f0029b57-ef5a-4190-8f66-fe6799580f41 None None] MFA Rules not processed for user `113c637cb4b543b19e72700a4be5f7be`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:50.265747 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f0029b57-ef5a-4190-8f66-fe6799580f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:50.306827 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:50.366718 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1304/2595] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:50.367849 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09015ccd-ade0-4d00-ae99-178083eddd04 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:50.445798 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09015ccd-ade0-4d00-ae99-178083eddd04 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:50.451339 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e712816-7a28-4f22-97b4-c91e2269ebf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:50.451939 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e712816-7a28-4f22-97b4-c91e2269ebf4 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:50.452213 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e712816-7a28-4f22-97b4-c91e2269ebf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:50.452479 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e712816-7a28-4f22-97b4-c91e2269ebf4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:50.453113 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e712816-7a28-4f22-97b4-c91e2269ebf4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:50.455023 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e712816-7a28-4f22-97b4-c91e2269ebf4 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:50.491218 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1292/2596] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:48:50 2022] POST /identity/v3/projects => generated 381 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:48:50.494832 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0029b57-ef5a-4190-8f66-fe6799580f41 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:50.506899 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0029b57-ef5a-4190-8f66-fe6799580f41 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:50.578610 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed5625c-5698-408b-9573-c66b057b0680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:50.581062 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5625c-5698-408b-9573-c66b057b0680 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:50.581353 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5625c-5698-408b-9573-c66b057b0680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:50.581627 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed5625c-5698-408b-9573-c66b057b0680 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:50.582330 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed5625c-5698-408b-9573-c66b057b0680 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:50.586548 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed5625c-5698-408b-9573-c66b057b0680 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:50.612395 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1305/2597] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:48:50 2022] POST /identity/v3/users => generated 297 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:48:50.617398 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e712816-7a28-4f22-97b4-c91e2269ebf4 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:50.629208 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e712816-7a28-4f22-97b4-c91e2269ebf4 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:50.634721 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf4803-c871-48ba-b610-37a96850d733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:50.635333 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf4803-c871-48ba-b610-37a96850d733 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:50.635611 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf4803-c871-48ba-b610-37a96850d733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:50.636376 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf4803-c871-48ba-b610-37a96850d733 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:50.637032 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbf4803-c871-48ba-b610-37a96850d733 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:50.638968 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbf4803-c871-48ba-b610-37a96850d733 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:50.650838 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1293/2598] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:48:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:48:50.655376 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed5625c-5698-408b-9573-c66b057b0680 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:50.666614 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed5625c-5698-408b-9573-c66b057b0680 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:50.671466 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b10253-919d-489c-8199-0496e73ff6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:50.672122 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b10253-919d-489c-8199-0496e73ff6ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:50.672395 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b10253-919d-489c-8199-0496e73ff6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:50.672668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b10253-919d-489c-8199-0496e73ff6ac admin admin] PATH_INFO: `/v3/projects/84be26d6f65442ff9fa7b0d866fadc32/users/3333a98ff56d4cd3b8b9c24d5d34fad9/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:50.690976 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b10253-919d-489c-8199-0496e73ff6ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=84be26d6f65442ff9fa7b0d866fadc32, user_id=3333a98ff56d4cd3b8b9c24d5d34fad9, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:50.694112 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b10253-919d-489c-8199-0496e73ff6ac admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:50.707333 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1306/2599] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:48:50 2022] PUT /identity/v3/projects/84be26d6f65442ff9fa7b0d866fadc32/users/3333a98ff56d4cd3b8b9c24d5d34fad9/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:50.711413 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf4803-c871-48ba-b610-37a96850d733 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:50.724701 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf4803-c871-48ba-b610-37a96850d733 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:50.801624 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38e2ad4-0bd9-4c2b-ba20-97cca862e7fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:50.803041 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e2ad4-0bd9-4c2b-ba20-97cca862e7fd admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:50.803678 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e2ad4-0bd9-4c2b-ba20-97cca862e7fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:50.804202 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e2ad4-0bd9-4c2b-ba20-97cca862e7fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:50.806124 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38e2ad4-0bd9-4c2b-ba20-97cca862e7fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:50.808005 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38e2ad4-0bd9-4c2b-ba20-97cca862e7fd admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:50.822544 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1294/2600] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:48:50 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:48:50.823709 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b10253-919d-489c-8199-0496e73ff6ac admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:50.837228 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b10253-919d-489c-8199-0496e73ff6ac admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:50.842348 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f0b66-ca24-42aa-964e-41592cfaffb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:50.843004 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f0b66-ca24-42aa-964e-41592cfaffb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:50.843286 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f0b66-ca24-42aa-964e-41592cfaffb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:50.843602 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f0b66-ca24-42aa-964e-41592cfaffb5 admin admin] PATH_INFO: `/v3/projects/84be26d6f65442ff9fa7b0d866fadc32/users/3333a98ff56d4cd3b8b9c24d5d34fad9/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:50.846490 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6f0b66-ca24-42aa-964e-41592cfaffb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84be26d6f65442ff9fa7b0d866fadc32, user_id=3333a98ff56d4cd3b8b9c24d5d34fad9, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:50.849871 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6f0b66-ca24-42aa-964e-41592cfaffb5 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:50.866661 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1307/2601] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:48:50 2022] PUT /identity/v3/projects/84be26d6f65442ff9fa7b0d866fadc32/users/3333a98ff56d4cd3b8b9c24d5d34fad9/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:50.878713 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af06e15-fce6-4153-89ba-c9d5380b5b3f None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:50.878999 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af06e15-fce6-4153-89ba-c9d5380b5b3f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:50.879275 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af06e15-fce6-4153-89ba-c9d5380b5b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:50.911826 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-2af06e15-fce6-4153-89ba-c9d5380b5b3f None None] MFA Rules not processed for user `3333a98ff56d4cd3b8b9c24d5d34fad9`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:50.913034 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-2af06e15-fce6-4153-89ba-c9d5380b5b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:50.943993 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:50.996329 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1295/2602] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:50 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:51.008162 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f0b66-ca24-42aa-964e-41592cfaffb5 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:51.009830 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-fa6f0b66-ca24-42aa-964e-41592cfaffb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:51.071086 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f0b66-ca24-42aa-964e-41592cfaffb5 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:51.075341 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc0aa51-45f4-4747-a6d6-2c6c6a4a7cbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:51.075946 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0aa51-45f4-4747-a6d6-2c6c6a4a7cbf service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:51.076236 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0aa51-45f4-4747-a6d6-2c6c6a4a7cbf service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:51.076500 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0aa51-45f4-4747-a6d6-2c6c6a4a7cbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:51.088066 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc0aa51-45f4-4747-a6d6-2c6c6a4a7cbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:51.089499 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc0aa51-45f4-4747-a6d6-2c6c6a4a7cbf service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:51.100022 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1308/2603] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:48:51 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:48:51.212256 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af06e15-fce6-4153-89ba-c9d5380b5b3f None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:51.221894 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af06e15-fce6-4153-89ba-c9d5380b5b3f None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:51.226027 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a450ec9-3c4a-4f17-aa32-59d10181ed5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:51.226661 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a450ec9-3c4a-4f17-aa32-59d10181ed5a service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:51.226936 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a450ec9-3c4a-4f17-aa32-59d10181ed5a service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:51.227204 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a450ec9-3c4a-4f17-aa32-59d10181ed5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:51.287356 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a450ec9-3c4a-4f17-aa32-59d10181ed5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:51.288833 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a450ec9-3c4a-4f17-aa32-59d10181ed5a service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:51.300415 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1296/2604] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:48:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:48:51.362879 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc0aa51-45f4-4747-a6d6-2c6c6a4a7cbf service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:51.365410 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9dc0aa51-45f4-4747-a6d6-2c6c6a4a7cbf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:51.399049 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:51.449781 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc0aa51-45f4-4747-a6d6-2c6c6a4a7cbf service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:51.454709 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3020f88a-963b-4376-8835-ae366d6d80f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:51.455375 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3020f88a-963b-4376-8835-ae366d6d80f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:51.455657 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3020f88a-963b-4376-8835-ae366d6d80f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:51.455933 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3020f88a-963b-4376-8835-ae366d6d80f4 admin admin] PATH_INFO: `/v3/users/48f6632d9fe24aeeb9af6955f85a4c0c` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:51.457049 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3020f88a-963b-4376-8835-ae366d6d80f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48f6632d9fe24aeeb9af6955f85a4c0c)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:51.459226 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3020f88a-963b-4376-8835-ae366d6d80f4 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:51.508547 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3020f88a-963b-4376-8835-ae366d6d80f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48f6632d9fe24aeeb9af6955f85a4c0c'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:51.509211 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3020f88a-963b-4376-8835-ae366d6d80f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48f6632d9fe24aeeb9af6955f85a4c0c'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:51.524197 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3020f88a-963b-4376-8835-ae366d6d80f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48f6632d9fe24aeeb9af6955f85a4c0c'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:51.531679 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3020f88a-963b-4376-8835-ae366d6d80f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48f6632d9fe24aeeb9af6955f85a4c0c'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:51.540132 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-3020f88a-963b-4376-8835-ae366d6d80f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48f6632d9fe24aeeb9af6955f85a4c0c'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:51.546663 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1309/2605] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:48:51 2022] DELETE /identity/v3/users/48f6632d9fe24aeeb9af6955f85a4c0c => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:51.729363 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a450ec9-3c4a-4f17-aa32-59d10181ed5a service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:51.740268 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a450ec9-3c4a-4f17-aa32-59d10181ed5a service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:51.795883 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b49485-56a6-4a33-86a3-bd2d78cf1bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:51.796500 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b49485-56a6-4a33-86a3-bd2d78cf1bca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:51.796773 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b49485-56a6-4a33-86a3-bd2d78cf1bca admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:51.797035 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b49485-56a6-4a33-86a3-bd2d78cf1bca admin admin] PATH_INFO: `/v3/projects/e14714c2f8ba4d1b9b009e829547f1c7` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:51.798141 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b49485-56a6-4a33-86a3-bd2d78cf1bca admin admin] RBAC: Authorizing `identity:delete_project(project_id=e14714c2f8ba4d1b9b009e829547f1c7)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:51.800004 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b49485-56a6-4a33-86a3-bd2d78cf1bca admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:51.846199 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-53b49485-56a6-4a33-86a3-bd2d78cf1bca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e14714c2f8ba4d1b9b009e829547f1c7', 'request_id': 'req-53b49485-56a6-4a33-86a3-bd2d78cf1bca'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:51.853388 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-53b49485-56a6-4a33-86a3-bd2d78cf1bca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e14714c2f8ba4d1b9b009e829547f1c7', 'request_id': 'req-53b49485-56a6-4a33-86a3-bd2d78cf1bca'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:51.859543 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-53b49485-56a6-4a33-86a3-bd2d78cf1bca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e14714c2f8ba4d1b9b009e829547f1c7', 'request_id': 'req-53b49485-56a6-4a33-86a3-bd2d78cf1bca'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:51.861495 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:51.875102 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-53b49485-56a6-4a33-86a3-bd2d78cf1bca admin admin] The token cache is being invalidate because project e14714c2f8ba4d1b9b009e829547f1c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:48:51.875394 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-53b49485-56a6-4a33-86a3-bd2d78cf1bca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:51.876036 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-53b49485-56a6-4a33-86a3-bd2d78cf1bca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:51.877544 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1297/2606] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:48:51 2022] DELETE /identity/v3/projects/e14714c2f8ba4d1b9b009e829547f1c7 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:51.882364 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3020f88a-963b-4376-8835-ae366d6d80f4 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:51.883795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3020f88a-963b-4376-8835-ae366d6d80f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:51.897875 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3020f88a-963b-4376-8835-ae366d6d80f4 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:51.955964 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e1cb97-55a1-4178-ad7a-7db5610a559c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:51.956581 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e1cb97-55a1-4178-ad7a-7db5610a559c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:51.956862 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e1cb97-55a1-4178-ad7a-7db5610a559c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:51.957160 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e1cb97-55a1-4178-ad7a-7db5610a559c admin admin] PATH_INFO: `/v3/users/a3e63ada65b54f1fa28d6a5b6eeb9da3` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:51.958300 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e1cb97-55a1-4178-ad7a-7db5610a559c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3e63ada65b54f1fa28d6a5b6eeb9da3)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:51.960227 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e1cb97-55a1-4178-ad7a-7db5610a559c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:52.005980 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:52.009141 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b6e1cb97-55a1-4178-ad7a-7db5610a559c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3e63ada65b54f1fa28d6a5b6eeb9da3'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:52.010272 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b6e1cb97-55a1-4178-ad7a-7db5610a559c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3e63ada65b54f1fa28d6a5b6eeb9da3'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:52.025748 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b6e1cb97-55a1-4178-ad7a-7db5610a559c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3e63ada65b54f1fa28d6a5b6eeb9da3'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:52.034265 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b6e1cb97-55a1-4178-ad7a-7db5610a559c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3e63ada65b54f1fa28d6a5b6eeb9da3'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:52.041655 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-b6e1cb97-55a1-4178-ad7a-7db5610a559c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3e63ada65b54f1fa28d6a5b6eeb9da3'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:52.047761 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1310/2607] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:48:51 2022] DELETE /identity/v3/users/a3e63ada65b54f1fa28d6a5b6eeb9da3 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:52.216553 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b49485-56a6-4a33-86a3-bd2d78cf1bca admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:52.217986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-53b49485-56a6-4a33-86a3-bd2d78cf1bca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:52.230605 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b49485-56a6-4a33-86a3-bd2d78cf1bca admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:52.289501 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c504c-a484-49af-a384-6359797e075e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:52.290187 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c504c-a484-49af-a384-6359797e075e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:52.290460 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c504c-a484-49af-a384-6359797e075e admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:52.290726 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c504c-a484-49af-a384-6359797e075e admin admin] PATH_INFO: `/v3/projects/e1d48929f99c4f00aa13fb134d1072a1` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:52.291823 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39c504c-a484-49af-a384-6359797e075e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1d48929f99c4f00aa13fb134d1072a1)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:52.293699 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39c504c-a484-49af-a384-6359797e075e admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:52.339016 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d39c504c-a484-49af-a384-6359797e075e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1d48929f99c4f00aa13fb134d1072a1', 'request_id': 'req-d39c504c-a484-49af-a384-6359797e075e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:52.345304 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d39c504c-a484-49af-a384-6359797e075e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1d48929f99c4f00aa13fb134d1072a1', 'request_id': 'req-d39c504c-a484-49af-a384-6359797e075e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:52.351825 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d39c504c-a484-49af-a384-6359797e075e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1d48929f99c4f00aa13fb134d1072a1', 'request_id': 'req-d39c504c-a484-49af-a384-6359797e075e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:52.361862 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d39c504c-a484-49af-a384-6359797e075e admin admin] The token cache is being invalidate because project e1d48929f99c4f00aa13fb134d1072a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:48:52.362158 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d39c504c-a484-49af-a384-6359797e075e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:52.362806 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d39c504c-a484-49af-a384-6359797e075e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:52.365353 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1298/2608] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:48:52 2022] DELETE /identity/v3/projects/e1d48929f99c4f00aa13fb134d1072a1 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:52.370554 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6123c-dcee-4edd-ac73-8bac96572e9c None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:52.370844 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6123c-dcee-4edd-ac73-8bac96572e9c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:52.371119 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6123c-dcee-4edd-ac73-8bac96572e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:52.390619 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-bff6123c-dcee-4edd-ac73-8bac96572e9c None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:52.391807 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-bff6123c-dcee-4edd-ac73-8bac96572e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:52.449232 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1311/2609] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:52.514088 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2131b558-2056-424f-a328-72ecfc7fa07a None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:52.514368 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2131b558-2056-424f-a328-72ecfc7fa07a None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:52.514638 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2131b558-2056-424f-a328-72ecfc7fa07a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:52.534630 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-2131b558-2056-424f-a328-72ecfc7fa07a None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:52.535995 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-2131b558-2056-424f-a328-72ecfc7fa07a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:52.543643 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1299/2610] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:52.547888 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff6123c-dcee-4edd-ac73-8bac96572e9c None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:52.557432 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff6123c-dcee-4edd-ac73-8bac96572e9c None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:52.561976 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a5c324-a666-4f93-ad69-36b735644b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:52.562568 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5c324-a666-4f93-ad69-36b735644b02 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:52.562849 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5c324-a666-4f93-ad69-36b735644b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:52.563121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5c324-a666-4f93-ad69-36b735644b02 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:52.563751 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a5c324-a666-4f93-ad69-36b735644b02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:48:52.564052 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a5c324-a666-4f93-ad69-36b735644b02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:52.565635 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a5c324-a666-4f93-ad69-36b735644b02 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:52.577362 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1312/2611] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:48:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:48:52.581585 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2131b558-2056-424f-a328-72ecfc7fa07a None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:52.591375 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2131b558-2056-424f-a328-72ecfc7fa07a None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:52.595989 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21245d1-99a9-46de-9418-187d49f14c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:52.596570 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21245d1-99a9-46de-9418-187d49f14c09 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:52.596849 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21245d1-99a9-46de-9418-187d49f14c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:52.597120 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21245d1-99a9-46de-9418-187d49f14c09 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:52.597742 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21245d1-99a9-46de-9418-187d49f14c09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:52.599671 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21245d1-99a9-46de-9418-187d49f14c09 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:52.631108 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1300/2612] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:48:52 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:48:52.632426 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a5c324-a666-4f93-ad69-36b735644b02 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:52.642437 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a5c324-a666-4f93-ad69-36b735644b02 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:52.702275 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8553bd46-64ac-4b9d-81bd-675b2c6dcff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:52.702957 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553bd46-64ac-4b9d-81bd-675b2c6dcff4 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:52.703311 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553bd46-64ac-4b9d-81bd-675b2c6dcff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:52.703675 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553bd46-64ac-4b9d-81bd-675b2c6dcff4 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:52.704402 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8553bd46-64ac-4b9d-81bd-675b2c6dcff4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:52.706341 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8553bd46-64ac-4b9d-81bd-675b2c6dcff4 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:52.730678 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1313/2613] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:48:52 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:48:52.731796 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21245d1-99a9-46de-9418-187d49f14c09 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:52.743079 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21245d1-99a9-46de-9418-187d49f14c09 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:52.748661 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5831ba-c2e9-431b-b903-ccb64a924619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:52.749275 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5831ba-c2e9-431b-b903-ccb64a924619 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:52.749652 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5831ba-c2e9-431b-b903-ccb64a924619 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:52.749959 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5831ba-c2e9-431b-b903-ccb64a924619 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:52.750610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5831ba-c2e9-431b-b903-ccb64a924619 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:52.752188 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5831ba-c2e9-431b-b903-ccb64a924619 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:52.766779 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1301/2614] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:48:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:48:52.767456 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8553bd46-64ac-4b9d-81bd-675b2c6dcff4 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:52.777422 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8553bd46-64ac-4b9d-81bd-675b2c6dcff4 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:52.782567 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf2f74-9693-4d89-a42b-230a86b06db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:52.783058 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf2f74-9693-4d89-a42b-230a86b06db2 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:52.783343 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf2f74-9693-4d89-a42b-230a86b06db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:52.783616 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf2f74-9693-4d89-a42b-230a86b06db2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:52.784253 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbf2f74-9693-4d89-a42b-230a86b06db2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:52.785927 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbf2f74-9693-4d89-a42b-230a86b06db2 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:52.797470 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1314/2615] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:48:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:48:52.802125 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5831ba-c2e9-431b-b903-ccb64a924619 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:52.812350 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5831ba-c2e9-431b-b903-ccb64a924619 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:52.817789 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca33251-fb36-404e-ab9e-81343a996666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:52.818479 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca33251-fb36-404e-ab9e-81343a996666 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:52.818751 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca33251-fb36-404e-ab9e-81343a996666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:52.819016 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca33251-fb36-404e-ab9e-81343a996666 admin admin] PATH_INFO: `/v3/projects/3e09ad6c880d43ba92ffe0d12699d7e4/users/1b4e022e090a4979a54e4de5cb44bc86/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:52.834050 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca33251-fb36-404e-ab9e-81343a996666 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e09ad6c880d43ba92ffe0d12699d7e4, user_id=1b4e022e090a4979a54e4de5cb44bc86, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:52.837368 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca33251-fb36-404e-ab9e-81343a996666 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:52.853751 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1302/2616] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:48:52 2022] PUT /identity/v3/projects/3e09ad6c880d43ba92ffe0d12699d7e4/users/1b4e022e090a4979a54e4de5cb44bc86/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:52.865752 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cda2d64-fa7f-4bdf-a4e2-c23ac7514cc0 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:52.866086 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cda2d64-fa7f-4bdf-a4e2-c23ac7514cc0 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:52.866409 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cda2d64-fa7f-4bdf-a4e2-c23ac7514cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:52.896829 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-1cda2d64-fa7f-4bdf-a4e2-c23ac7514cc0 None None] MFA Rules not processed for user `1b4e022e090a4979a54e4de5cb44bc86`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:52.898060 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1cda2d64-fa7f-4bdf-a4e2-c23ac7514cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:52.910421 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:52.968424 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1315/2617] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:52.972919 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca33251-fb36-404e-ab9e-81343a996666 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:52.993611 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:53.042926 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca33251-fb36-404e-ab9e-81343a996666 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:53.048641 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f3e14e-e5c0-4c97-b87a-2224c2a2f17a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:53.049334 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f3e14e-e5c0-4c97-b87a-2224c2a2f17a admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:53.049577 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f3e14e-e5c0-4c97-b87a-2224c2a2f17a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:53.049958 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f3e14e-e5c0-4c97-b87a-2224c2a2f17a admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:53.050629 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f3e14e-e5c0-4c97-b87a-2224c2a2f17a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:53.053621 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f3e14e-e5c0-4c97-b87a-2224c2a2f17a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:53.086117 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1303/2618] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:48:52 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:48:53.090163 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cda2d64-fa7f-4bdf-a4e2-c23ac7514cc0 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:53.101926 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cda2d64-fa7f-4bdf-a4e2-c23ac7514cc0 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:53.157271 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead92199-1665-4e1b-b50f-eeb48fa8d8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:53.157935 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead92199-1665-4e1b-b50f-eeb48fa8d8f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:53.158219 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead92199-1665-4e1b-b50f-eeb48fa8d8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:53.158492 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead92199-1665-4e1b-b50f-eeb48fa8d8f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:53.159141 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead92199-1665-4e1b-b50f-eeb48fa8d8f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:53.161593 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead92199-1665-4e1b-b50f-eeb48fa8d8f9 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:53.182283 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1316/2619] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:48:53 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:48:53.186186 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f3e14e-e5c0-4c97-b87a-2224c2a2f17a admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:53.196229 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f3e14e-e5c0-4c97-b87a-2224c2a2f17a admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:53.200719 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf680a5-f621-4c54-91e4-cf6c5db14186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:53.201298 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf680a5-f621-4c54-91e4-cf6c5db14186 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:53.201570 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf680a5-f621-4c54-91e4-cf6c5db14186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:53.201835 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf680a5-f621-4c54-91e4-cf6c5db14186 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:53.202488 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf680a5-f621-4c54-91e4-cf6c5db14186 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:53.204041 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf680a5-f621-4c54-91e4-cf6c5db14186 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:53.214291 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1304/2620] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:48:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:48:53.218176 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead92199-1665-4e1b-b50f-eeb48fa8d8f9 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:53.228434 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead92199-1665-4e1b-b50f-eeb48fa8d8f9 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:53.234533 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da84b192-ef6e-4728-ac85-df43deb8d8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:53.235320 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da84b192-ef6e-4728-ac85-df43deb8d8a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:53.235599 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da84b192-ef6e-4728-ac85-df43deb8d8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:53.235866 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da84b192-ef6e-4728-ac85-df43deb8d8a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:53.236503 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da84b192-ef6e-4728-ac85-df43deb8d8a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:53.238573 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da84b192-ef6e-4728-ac85-df43deb8d8a4 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:53.250755 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1317/2621] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:48:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:48:53.254159 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf680a5-f621-4c54-91e4-cf6c5db14186 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:53.265962 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf680a5-f621-4c54-91e4-cf6c5db14186 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:53.270887 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afaf2cf-8764-480d-8a78-0d102a4cfeda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:53.271627 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afaf2cf-8764-480d-8a78-0d102a4cfeda admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:53.271896 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afaf2cf-8764-480d-8a78-0d102a4cfeda admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:53.272164 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afaf2cf-8764-480d-8a78-0d102a4cfeda admin admin] PATH_INFO: `/v3/projects/51de70265e5243d2b0d629631e1ba468/users/af14b215a46c4176a693f77b8f70da4e/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:53.287781 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afaf2cf-8764-480d-8a78-0d102a4cfeda admin admin] RBAC: Authorizing `identity:create_grant(project_id=51de70265e5243d2b0d629631e1ba468, user_id=af14b215a46c4176a693f77b8f70da4e, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:53.290430 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afaf2cf-8764-480d-8a78-0d102a4cfeda admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:53.304469 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1305/2622] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:48:53 2022] PUT /identity/v3/projects/51de70265e5243d2b0d629631e1ba468/users/af14b215a46c4176a693f77b8f70da4e/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:53.314706 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c2808-fa6a-445f-be31-4971008c822e None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:53.314991 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c2808-fa6a-445f-be31-4971008c822e None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:53.315260 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46c2808-fa6a-445f-be31-4971008c822e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:53.342724 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-b46c2808-fa6a-445f-be31-4971008c822e None None] MFA Rules not processed for user `af14b215a46c4176a693f77b8f70da4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:53.343926 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b46c2808-fa6a-445f-be31-4971008c822e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:53.411383 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1318/2623] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:53.415960 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afaf2cf-8764-480d-8a78-0d102a4cfeda admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:53.476407 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afaf2cf-8764-480d-8a78-0d102a4cfeda admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:53.481039 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86fa6b1-2758-4a64-aba4-6952788058dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:53.481575 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86fa6b1-2758-4a64-aba4-6952788058dc admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:53.481886 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86fa6b1-2758-4a64-aba4-6952788058dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:53.482153 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86fa6b1-2758-4a64-aba4-6952788058dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:53.482769 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86fa6b1-2758-4a64-aba4-6952788058dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:53.484640 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86fa6b1-2758-4a64-aba4-6952788058dc admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:53.510051 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1306/2624] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:48:53 2022] POST /identity/v3/projects => generated 383 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:48:53.514242 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c2808-fa6a-445f-be31-4971008c822e None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:53.525398 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46c2808-fa6a-445f-be31-4971008c822e None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:53.545933 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:53.586050 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b2c6e6-d57f-47be-bfa4-9a1f5f4553c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:53.586661 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2c6e6-d57f-47be-bfa4-9a1f5f4553c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:53.586937 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2c6e6-d57f-47be-bfa4-9a1f5f4553c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:53.587234 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b2c6e6-d57f-47be-bfa4-9a1f5f4553c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:53.587859 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b2c6e6-d57f-47be-bfa4-9a1f5f4553c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:53.589777 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b2c6e6-d57f-47be-bfa4-9a1f5f4553c2 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:53.611080 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1319/2625] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:48:53 2022] POST /identity/v3/users => generated 298 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:48:53.615454 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86fa6b1-2758-4a64-aba4-6952788058dc admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:53.625628 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86fa6b1-2758-4a64-aba4-6952788058dc admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:53.630424 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015e315-33d6-4f25-8647-432c0de40467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:53.631011 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015e315-33d6-4f25-8647-432c0de40467 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:53.631285 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015e315-33d6-4f25-8647-432c0de40467 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:53.631552 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015e315-33d6-4f25-8647-432c0de40467 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:53.632179 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d015e315-33d6-4f25-8647-432c0de40467 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:53.633779 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d015e315-33d6-4f25-8647-432c0de40467 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:53.644073 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1307/2626] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:48:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:48:53.648570 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b2c6e6-d57f-47be-bfa4-9a1f5f4553c2 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:53.658356 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b2c6e6-d57f-47be-bfa4-9a1f5f4553c2 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:53.663829 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a4915-a472-4998-95ea-ef2a12bc40ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:53.664471 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a4915-a472-4998-95ea-ef2a12bc40ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:53.664818 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a4915-a472-4998-95ea-ef2a12bc40ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:53.665106 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a4915-a472-4998-95ea-ef2a12bc40ff admin admin] PATH_INFO: `/v3/projects/0567ed059c4640c485c043332e33a10e/users/eb8f4cea637f4141b7eeec0f5d257158/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:53.681241 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381a4915-a472-4998-95ea-ef2a12bc40ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=0567ed059c4640c485c043332e33a10e, user_id=eb8f4cea637f4141b7eeec0f5d257158, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:53.683940 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381a4915-a472-4998-95ea-ef2a12bc40ff admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:53.696298 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1320/2627] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:48:53 2022] PUT /identity/v3/projects/0567ed059c4640c485c043332e33a10e/users/eb8f4cea637f4141b7eeec0f5d257158/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:48:53.700273 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015e315-33d6-4f25-8647-432c0de40467 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:53.710527 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015e315-33d6-4f25-8647-432c0de40467 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:53.749163 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:53.765493 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60028a7d-e387-4eae-a003-bb6f15ce5869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:53.766303 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60028a7d-e387-4eae-a003-bb6f15ce5869 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:53.766436 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60028a7d-e387-4eae-a003-bb6f15ce5869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:53.766658 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60028a7d-e387-4eae-a003-bb6f15ce5869 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:53.767389 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60028a7d-e387-4eae-a003-bb6f15ce5869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:53.769236 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60028a7d-e387-4eae-a003-bb6f15ce5869 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:53.779368 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1308/2628] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:48:53 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:48:53.783685 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a4915-a472-4998-95ea-ef2a12bc40ff admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:53.794441 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a4915-a472-4998-95ea-ef2a12bc40ff admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:53.798999 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be99bde-f752-4f63-b91a-1c7ba2ccccea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:53.799663 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be99bde-f752-4f63-b91a-1c7ba2ccccea admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:53.799945 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be99bde-f752-4f63-b91a-1c7ba2ccccea admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:53.800220 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be99bde-f752-4f63-b91a-1c7ba2ccccea admin admin] PATH_INFO: `/v3/projects/0567ed059c4640c485c043332e33a10e/users/eb8f4cea637f4141b7eeec0f5d257158/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:53.802245 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be99bde-f752-4f63-b91a-1c7ba2ccccea admin admin] RBAC: Authorizing `identity:create_grant(project_id=0567ed059c4640c485c043332e33a10e, user_id=eb8f4cea637f4141b7eeec0f5d257158, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:53.804831 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be99bde-f752-4f63-b91a-1c7ba2ccccea admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:53.814731 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1321/2629] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:48:53 2022] PUT /identity/v3/projects/0567ed059c4640c485c043332e33a10e/users/eb8f4cea637f4141b7eeec0f5d257158/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:53.826417 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedbc7de-433a-436f-98c0-7ad3a98b6bf1 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:53.826698 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedbc7de-433a-436f-98c0-7ad3a98b6bf1 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:53.826970 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedbc7de-433a-436f-98c0-7ad3a98b6bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:53.855183 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-eedbc7de-433a-436f-98c0-7ad3a98b6bf1 None None] MFA Rules not processed for user `eb8f4cea637f4141b7eeec0f5d257158`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:53.856378 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-eedbc7de-433a-436f-98c0-7ad3a98b6bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:53.926234 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1309/2630] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:53 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:53.931200 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be99bde-f752-4f63-b91a-1c7ba2ccccea admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:53.933260 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2be99bde-f752-4f63-b91a-1c7ba2ccccea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:53.937492 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedbc7de-433a-436f-98c0-7ad3a98b6bf1 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:53.938937 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-eedbc7de-433a-436f-98c0-7ad3a98b6bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:53.996109 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedbc7de-433a-436f-98c0-7ad3a98b6bf1 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:53.998519 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be99bde-f752-4f63-b91a-1c7ba2ccccea admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:53.999745 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a16ad7c-4dd7-4214-bde3-724735e21955 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:54.000283 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a16ad7c-4dd7-4214-bde3-724735e21955 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:54.000559 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a16ad7c-4dd7-4214-bde3-724735e21955 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:54.000826 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a16ad7c-4dd7-4214-bde3-724735e21955 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:54.003221 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9a9f9f-f687-4088-8db3-28b228eee39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:54.003838 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a9f9f-f687-4088-8db3-28b228eee39a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:54.004121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a9f9f-f687-4088-8db3-28b228eee39a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:54.004945 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a9f9f-f687-4088-8db3-28b228eee39a admin admin] PATH_INFO: `/v3/users/113c637cb4b543b19e72700a4be5f7be` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:54.006190 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9a9f9f-f687-4088-8db3-28b228eee39a admin admin] RBAC: Authorizing `identity:delete_user(user_id=113c637cb4b543b19e72700a4be5f7be)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:54.008102 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9a9f9f-f687-4088-8db3-28b228eee39a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:54.056425 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-cf9a9f9f-f687-4088-8db3-28b228eee39a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '113c637cb4b543b19e72700a4be5f7be'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:54.057071 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-cf9a9f9f-f687-4088-8db3-28b228eee39a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '113c637cb4b543b19e72700a4be5f7be'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:54.060823 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a16ad7c-4dd7-4214-bde3-724735e21955 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:54.062348 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a16ad7c-4dd7-4214-bde3-724735e21955 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:54.069680 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-cf9a9f9f-f687-4088-8db3-28b228eee39a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '113c637cb4b543b19e72700a4be5f7be'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:54.073770 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1310/2631] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:48:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:48:54.077599 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-cf9a9f9f-f687-4088-8db3-28b228eee39a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '113c637cb4b543b19e72700a4be5f7be'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:54.084717 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:54.087989 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-cf9a9f9f-f687-4088-8db3-28b228eee39a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '113c637cb4b543b19e72700a4be5f7be'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:54.094917 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1322/2632] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:48:53 2022] DELETE /identity/v3/users/113c637cb4b543b19e72700a4be5f7be => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:54.099425 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5f5fc-db17-4238-aa9a-68a82b67d390 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:54.099709 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5f5fc-db17-4238-aa9a-68a82b67d390 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:54.100002 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5f5fc-db17-4238-aa9a-68a82b67d390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:54.121009 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-5ad5f5fc-db17-4238-aa9a-68a82b67d390 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:54.122298 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-5ad5f5fc-db17-4238-aa9a-68a82b67d390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:54.135810 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:54.192937 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1311/2633] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:48:54.202632 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9a9f9f-f687-4088-8db3-28b228eee39a admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:54.204035 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-cf9a9f9f-f687-4088-8db3-28b228eee39a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:54.268542 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9a9f9f-f687-4088-8db3-28b228eee39a admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:54.272327 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a629024-9ab3-47a7-84c4-209121d92340 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:54.272958 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a629024-9ab3-47a7-84c4-209121d92340 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:54.273232 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a629024-9ab3-47a7-84c4-209121d92340 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:54.273498 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a629024-9ab3-47a7-84c4-209121d92340 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:54.283669 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a629024-9ab3-47a7-84c4-209121d92340 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:54.285089 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a629024-9ab3-47a7-84c4-209121d92340 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:54.294814 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1323/2634] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:48:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:48:54.779330 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad5f5fc-db17-4238-aa9a-68a82b67d390 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:54.781647 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-5ad5f5fc-db17-4238-aa9a-68a82b67d390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:54.794581 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad5f5fc-db17-4238-aa9a-68a82b67d390 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:54.800179 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32178e3a-bda8-4adb-9ce2-344a80428251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:54.800894 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32178e3a-bda8-4adb-9ce2-344a80428251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:54.801500 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32178e3a-bda8-4adb-9ce2-344a80428251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:54.801834 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32178e3a-bda8-4adb-9ce2-344a80428251 admin admin] PATH_INFO: `/v3/projects/c0bb76f8f11445ad884b64dc0fdde8d6` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:54.803435 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32178e3a-bda8-4adb-9ce2-344a80428251 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0bb76f8f11445ad884b64dc0fdde8d6)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:54.805482 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32178e3a-bda8-4adb-9ce2-344a80428251 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:54.855312 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-32178e3a-bda8-4adb-9ce2-344a80428251 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0bb76f8f11445ad884b64dc0fdde8d6', 'request_id': 'req-32178e3a-bda8-4adb-9ce2-344a80428251'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:54.863702 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-32178e3a-bda8-4adb-9ce2-344a80428251 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0bb76f8f11445ad884b64dc0fdde8d6', 'request_id': 'req-32178e3a-bda8-4adb-9ce2-344a80428251'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:54.870460 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-32178e3a-bda8-4adb-9ce2-344a80428251 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0bb76f8f11445ad884b64dc0fdde8d6', 'request_id': 'req-32178e3a-bda8-4adb-9ce2-344a80428251'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:54.881616 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-32178e3a-bda8-4adb-9ce2-344a80428251 admin admin] The token cache is being invalidate because project c0bb76f8f11445ad884b64dc0fdde8d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:48:54.881930 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-32178e3a-bda8-4adb-9ce2-344a80428251 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:54.882575 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-32178e3a-bda8-4adb-9ce2-344a80428251 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:54.884267 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1312/2635] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:48:54 2022] DELETE /identity/v3/projects/c0bb76f8f11445ad884b64dc0fdde8d6 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:54.888209 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a629024-9ab3-47a7-84c4-209121d92340 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:54.898796 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a629024-9ab3-47a7-84c4-209121d92340 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:54.965298 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34ca2ef-e0ce-4cf9-b1e9-0ca53944464f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:54.966010 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34ca2ef-e0ce-4cf9-b1e9-0ca53944464f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:54.966237 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34ca2ef-e0ce-4cf9-b1e9-0ca53944464f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:54.966586 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34ca2ef-e0ce-4cf9-b1e9-0ca53944464f admin admin] PATH_INFO: `/v3/users/3333a98ff56d4cd3b8b9c24d5d34fad9` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:54.967890 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34ca2ef-e0ce-4cf9-b1e9-0ca53944464f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3333a98ff56d4cd3b8b9c24d5d34fad9)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:54.969905 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34ca2ef-e0ce-4cf9-b1e9-0ca53944464f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:55.017391 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e34ca2ef-e0ce-4cf9-b1e9-0ca53944464f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3333a98ff56d4cd3b8b9c24d5d34fad9'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:55.018130 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e34ca2ef-e0ce-4cf9-b1e9-0ca53944464f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3333a98ff56d4cd3b8b9c24d5d34fad9'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:55.030552 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e34ca2ef-e0ce-4cf9-b1e9-0ca53944464f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3333a98ff56d4cd3b8b9c24d5d34fad9'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:55.039298 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e34ca2ef-e0ce-4cf9-b1e9-0ca53944464f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3333a98ff56d4cd3b8b9c24d5d34fad9'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:55.046768 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e34ca2ef-e0ce-4cf9-b1e9-0ca53944464f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3333a98ff56d4cd3b8b9c24d5d34fad9'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:55.053068 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1324/2636] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:48:54 2022] DELETE /identity/v3/users/3333a98ff56d4cd3b8b9c24d5d34fad9 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:55.326615 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32178e3a-bda8-4adb-9ce2-344a80428251 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:55.328120 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-32178e3a-bda8-4adb-9ce2-344a80428251 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:55.342039 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32178e3a-bda8-4adb-9ce2-344a80428251 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:55.403529 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ad36e1-2b74-466a-96eb-3edbab559c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:55.404073 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad36e1-2b74-466a-96eb-3edbab559c23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:55.404343 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad36e1-2b74-466a-96eb-3edbab559c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:55.404607 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad36e1-2b74-466a-96eb-3edbab559c23 admin admin] PATH_INFO: `/v3/projects/84be26d6f65442ff9fa7b0d866fadc32` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:55.405696 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ad36e1-2b74-466a-96eb-3edbab559c23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84be26d6f65442ff9fa7b0d866fadc32)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:55.407616 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ad36e1-2b74-466a-96eb-3edbab559c23 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:55.451943 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:55.456071 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-71ad36e1-2b74-466a-96eb-3edbab559c23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84be26d6f65442ff9fa7b0d866fadc32', 'request_id': 'req-71ad36e1-2b74-466a-96eb-3edbab559c23'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:55.464427 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-71ad36e1-2b74-466a-96eb-3edbab559c23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84be26d6f65442ff9fa7b0d866fadc32', 'request_id': 'req-71ad36e1-2b74-466a-96eb-3edbab559c23'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:55.472823 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-71ad36e1-2b74-466a-96eb-3edbab559c23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84be26d6f65442ff9fa7b0d866fadc32', 'request_id': 'req-71ad36e1-2b74-466a-96eb-3edbab559c23'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:55.484147 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-71ad36e1-2b74-466a-96eb-3edbab559c23 admin admin] The token cache is being invalidate because project 84be26d6f65442ff9fa7b0d866fadc32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:48:55.484505 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-71ad36e1-2b74-466a-96eb-3edbab559c23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:55.485353 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-71ad36e1-2b74-466a-96eb-3edbab559c23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:55.486933 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1313/2637] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:48:55 2022] DELETE /identity/v3/projects/84be26d6f65442ff9fa7b0d866fadc32 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:55.492366 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb5fd0f-e75a-4951-8fd3-848ac2cfb896 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:55.492743 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb5fd0f-e75a-4951-8fd3-848ac2cfb896 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:55.493087 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb5fd0f-e75a-4951-8fd3-848ac2cfb896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:55.513970 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-4cb5fd0f-e75a-4951-8fd3-848ac2cfb896 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:55.515454 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4cb5fd0f-e75a-4951-8fd3-848ac2cfb896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:55.529400 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:55.585178 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1325/2638] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:55.592616 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0cf5b-e818-4244-8974-129323430af4 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:55.592915 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0cf5b-e818-4244-8974-129323430af4 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:55.593166 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0cf5b-e818-4244-8974-129323430af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:55.617024 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-e2c0cf5b-e818-4244-8974-129323430af4 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:55.618619 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e2c0cf5b-e818-4244-8974-129323430af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:55.626923 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1314/2639] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:55.635006 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb5fd0f-e75a-4951-8fd3-848ac2cfb896 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:55.642876 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c0cf5b-e818-4244-8974-129323430af4 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:55.644251 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e2c0cf5b-e818-4244-8974-129323430af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:55.646728 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb5fd0f-e75a-4951-8fd3-848ac2cfb896 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:55.652560 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3735d843-3983-4080-bf9b-6c561e3a9c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:55.653180 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3735d843-3983-4080-bf9b-6c561e3a9c18 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:55.653464 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3735d843-3983-4080-bf9b-6c561e3a9c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:55.653812 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3735d843-3983-4080-bf9b-6c561e3a9c18 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:55.654448 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3735d843-3983-4080-bf9b-6c561e3a9c18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:48:55.654758 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3735d843-3983-4080-bf9b-6c561e3a9c18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:55.656349 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3735d843-3983-4080-bf9b-6c561e3a9c18 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:55.669668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1326/2640] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:48:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:48:55.674323 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3735d843-3983-4080-bf9b-6c561e3a9c18 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:55.685465 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3735d843-3983-4080-bf9b-6c561e3a9c18 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:55.690457 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11026007-ddb9-474b-9183-090f9ac02f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:55.691370 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11026007-ddb9-474b-9183-090f9ac02f40 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:55.691370 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11026007-ddb9-474b-9183-090f9ac02f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:55.691582 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11026007-ddb9-474b-9183-090f9ac02f40 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:55.692214 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11026007-ddb9-474b-9183-090f9ac02f40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:55.694336 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11026007-ddb9-474b-9183-090f9ac02f40 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:55.708620 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c0cf5b-e818-4244-8974-129323430af4 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:55.712829 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4013a1-777a-4195-aa87-01356747fef3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:55.713683 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4013a1-777a-4195-aa87-01356747fef3 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:55.714016 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4013a1-777a-4195-aa87-01356747fef3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:55.714346 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4013a1-777a-4195-aa87-01356747fef3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:55.716484 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3c4013a1-777a-4195-aa87-01356747fef3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:55.723577 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1327/2641] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:48:55 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:48:55.727987 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11026007-ddb9-474b-9183-090f9ac02f40 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:55.783012 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4013a1-777a-4195-aa87-01356747fef3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:55.784525 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4013a1-777a-4195-aa87-01356747fef3 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:55.792487 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11026007-ddb9-474b-9183-090f9ac02f40 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:55.793671 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1315/2642] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:48:55 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:48:55.797157 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6405bba-5409-4a56-8a4f-963fca1582e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:55.797761 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6405bba-5409-4a56-8a4f-963fca1582e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:55.798074 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6405bba-5409-4a56-8a4f-963fca1582e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:55.798348 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6405bba-5409-4a56-8a4f-963fca1582e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:55.798963 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6405bba-5409-4a56-8a4f-963fca1582e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:55.800886 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6405bba-5409-4a56-8a4f-963fca1582e2 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:55.828117 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1328/2643] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:48:55 2022] POST /identity/v3/users => generated 294 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:48:55.829255 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4013a1-777a-4195-aa87-01356747fef3 service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:55.840909 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4013a1-777a-4195-aa87-01356747fef3 service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:55.845486 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b812793d-981f-49d9-bbe0-939b9dfb14be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:55.846117 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812793d-981f-49d9-bbe0-939b9dfb14be admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:55.846396 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812793d-981f-49d9-bbe0-939b9dfb14be admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:55.846661 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812793d-981f-49d9-bbe0-939b9dfb14be admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:55.847286 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b812793d-981f-49d9-bbe0-939b9dfb14be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:55.849444 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b812793d-981f-49d9-bbe0-939b9dfb14be admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:55.860306 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1316/2644] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:48:55.865186 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6405bba-5409-4a56-8a4f-963fca1582e2 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:55.876403 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6405bba-5409-4a56-8a4f-963fca1582e2 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:55.881767 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d232840b-15c9-462f-80a0-e117ec22207f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:55.882397 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d232840b-15c9-462f-80a0-e117ec22207f admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:55.882674 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d232840b-15c9-462f-80a0-e117ec22207f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:55.882944 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d232840b-15c9-462f-80a0-e117ec22207f admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:55.883572 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d232840b-15c9-462f-80a0-e117ec22207f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:55.885149 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d232840b-15c9-462f-80a0-e117ec22207f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:55.896079 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1329/2645] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:48:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:48:55.900709 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b812793d-981f-49d9-bbe0-939b9dfb14be admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:55.911786 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b812793d-981f-49d9-bbe0-939b9dfb14be admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:55.916919 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f347a-5120-46bb-8e0f-949448719fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:55.917629 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f347a-5120-46bb-8e0f-949448719fb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:55.918009 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f347a-5120-46bb-8e0f-949448719fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:55.918362 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f347a-5120-46bb-8e0f-949448719fb5 admin admin] PATH_INFO: `/v3/projects/483eae90fd2c4c40b21436b6d2c969f8/users/c0412b876536412f80b86653f19d1c47/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:55.933910 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f347a-5120-46bb-8e0f-949448719fb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=483eae90fd2c4c40b21436b6d2c969f8, user_id=c0412b876536412f80b86653f19d1c47, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:55.936590 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f347a-5120-46bb-8e0f-949448719fb5 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:55.950645 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1317/2646] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:48:55 2022] PUT /identity/v3/projects/483eae90fd2c4c40b21436b6d2c969f8/users/c0412b876536412f80b86653f19d1c47/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:48:55.961215 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa3eac2-1213-438f-9dbc-13c78766891c None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:55.961506 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa3eac2-1213-438f-9dbc-13c78766891c None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:55.961779 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa3eac2-1213-438f-9dbc-13c78766891c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:55.991304 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-4fa3eac2-1213-438f-9dbc-13c78766891c None None] MFA Rules not processed for user `c0412b876536412f80b86653f19d1c47`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:55.992616 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4fa3eac2-1213-438f-9dbc-13c78766891c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:56.043993 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:56.063984 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1330/2647] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:55 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:56.068592 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f347a-5120-46bb-8e0f-949448719fb5 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:56.123525 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:56.146407 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f347a-5120-46bb-8e0f-949448719fb5 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:56.151827 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8ef283-5115-450c-9c17-090b1768a5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:56.152503 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ef283-5115-450c-9c17-090b1768a5af admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:56.152853 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ef283-5115-450c-9c17-090b1768a5af admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:56.153194 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ef283-5115-450c-9c17-090b1768a5af admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:56.153904 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8ef283-5115-450c-9c17-090b1768a5af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:56.155897 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8ef283-5115-450c-9c17-090b1768a5af admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:56.190604 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1318/2648] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:48:56 2022] POST /identity/v3/projects => generated 387 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:48:56.192033 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa3eac2-1213-438f-9dbc-13c78766891c None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:56.203674 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa3eac2-1213-438f-9dbc-13c78766891c None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:56.298238 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aa317a-4906-4349-9b91-a40cea7f766e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:56.300503 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa317a-4906-4349-9b91-a40cea7f766e admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:56.300503 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa317a-4906-4349-9b91-a40cea7f766e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:56.300503 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa317a-4906-4349-9b91-a40cea7f766e admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:56.300503 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aa317a-4906-4349-9b91-a40cea7f766e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:56.303642 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aa317a-4906-4349-9b91-a40cea7f766e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:56.330864 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1331/2649] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:48:56 2022] POST /identity/v3/users => generated 300 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:48:56.334922 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8ef283-5115-450c-9c17-090b1768a5af admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:56.346344 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8ef283-5115-450c-9c17-090b1768a5af admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:56.351916 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12760071-80be-4761-b84c-01181f086cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:56.352398 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12760071-80be-4761-b84c-01181f086cca admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:56.352671 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12760071-80be-4761-b84c-01181f086cca admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:56.352934 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12760071-80be-4761-b84c-01181f086cca admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:56.353690 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12760071-80be-4761-b84c-01181f086cca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:56.355391 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12760071-80be-4761-b84c-01181f086cca admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:56.372325 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1319/2650] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:48:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:48:56.377199 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aa317a-4906-4349-9b91-a40cea7f766e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:56.391996 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aa317a-4906-4349-9b91-a40cea7f766e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:56.397944 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1194f5c4-c72b-4e36-8fd1-0e982b35fe3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:56.400757 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194f5c4-c72b-4e36-8fd1-0e982b35fe3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:56.400757 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194f5c4-c72b-4e36-8fd1-0e982b35fe3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:56.400757 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194f5c4-c72b-4e36-8fd1-0e982b35fe3b admin admin] PATH_INFO: `/v3/projects/c05c60ec4e9a4293aa90bca3e4e614c6/users/fe8da8fd3da34b9ab9a4480ec71b471f/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:56.402070 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12760071-80be-4761-b84c-01181f086cca admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:56.403879 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-12760071-80be-4761-b84c-01181f086cca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:56.418739 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1194f5c4-c72b-4e36-8fd1-0e982b35fe3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c05c60ec4e9a4293aa90bca3e4e614c6, user_id=fe8da8fd3da34b9ab9a4480ec71b471f, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:56.421498 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1194f5c4-c72b-4e36-8fd1-0e982b35fe3b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:56.437601 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1332/2651] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:48:56 2022] PUT /identity/v3/projects/c05c60ec4e9a4293aa90bca3e4e614c6/users/fe8da8fd3da34b9ab9a4480ec71b471f/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:56.441788 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1194f5c4-c72b-4e36-8fd1-0e982b35fe3b admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:56.473147 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12760071-80be-4761-b84c-01181f086cca admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:56.522368 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1194f5c4-c72b-4e36-8fd1-0e982b35fe3b admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:56.527031 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824cfaa5-d134-45eb-ae9d-0f27e1e34c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:56.527717 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824cfaa5-d134-45eb-ae9d-0f27e1e34c3e admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:56.528065 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824cfaa5-d134-45eb-ae9d-0f27e1e34c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:56.528540 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824cfaa5-d134-45eb-ae9d-0f27e1e34c3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:56.529178 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824cfaa5-d134-45eb-ae9d-0f27e1e34c3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:56.530736 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824cfaa5-d134-45eb-ae9d-0f27e1e34c3e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:56.533252 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a2aa62-5306-4ee1-89e6-07baaa2dfa34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:56.533998 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a2aa62-5306-4ee1-89e6-07baaa2dfa34 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:56.534663 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a2aa62-5306-4ee1-89e6-07baaa2dfa34 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:56.535027 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a2aa62-5306-4ee1-89e6-07baaa2dfa34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:56.539168 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-32a2aa62-5306-4ee1-89e6-07baaa2dfa34 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:56.542537 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1333/2652] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:48:56 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:48:56.546589 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824cfaa5-d134-45eb-ae9d-0f27e1e34c3e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:56.556466 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824cfaa5-d134-45eb-ae9d-0f27e1e34c3e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:56.561824 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f4ff97-8dd8-47e1-bffa-e7caf26da582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:56.562762 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4ff97-8dd8-47e1-bffa-e7caf26da582 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:56.563063 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4ff97-8dd8-47e1-bffa-e7caf26da582 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:56.563336 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4ff97-8dd8-47e1-bffa-e7caf26da582 admin admin] PATH_INFO: `/v3/projects/c05c60ec4e9a4293aa90bca3e4e614c6/users/fe8da8fd3da34b9ab9a4480ec71b471f/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:56.565391 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f4ff97-8dd8-47e1-bffa-e7caf26da582 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c05c60ec4e9a4293aa90bca3e4e614c6, user_id=fe8da8fd3da34b9ab9a4480ec71b471f, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:56.568896 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f4ff97-8dd8-47e1-bffa-e7caf26da582 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:56.583203 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1334/2653] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:48:56 2022] PUT /identity/v3/projects/c05c60ec4e9a4293aa90bca3e4e614c6/users/fe8da8fd3da34b9ab9a4480ec71b471f/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:48:56.594739 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e894584-4586-4c27-9710-ffe7c70d9b5a None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:56.595026 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e894584-4586-4c27-9710-ffe7c70d9b5a None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:56.595304 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e894584-4586-4c27-9710-ffe7c70d9b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:56.609941 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a2aa62-5306-4ee1-89e6-07baaa2dfa34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:56.611467 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a2aa62-5306-4ee1-89e6-07baaa2dfa34 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:56.622743 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1320/2654] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:48:56 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:48:56.627523 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-9e894584-4586-4c27-9710-ffe7c70d9b5a None None] MFA Rules not processed for user `fe8da8fd3da34b9ab9a4480ec71b471f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:56.628824 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9e894584-4586-4c27-9710-ffe7c70d9b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:56.664224 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:56.725011 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1335/2655] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:56 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:48:56.742206 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a2aa62-5306-4ee1-89e6-07baaa2dfa34 service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:56.748508 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a2aa62-5306-4ee1-89e6-07baaa2dfa34 service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:56.771627 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:56.806360 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56af633-20fc-4084-b016-bac8a5d95805 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:56.806987 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56af633-20fc-4084-b016-bac8a5d95805 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:56.807262 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56af633-20fc-4084-b016-bac8a5d95805 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:56.807617 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56af633-20fc-4084-b016-bac8a5d95805 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:56.820266 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56af633-20fc-4084-b016-bac8a5d95805 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:56.822014 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56af633-20fc-4084-b016-bac8a5d95805 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:56.832409 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1321/2656] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:48:56 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:48:56.920121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e894584-4586-4c27-9710-ffe7c70d9b5a None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:56.922877 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9e894584-4586-4c27-9710-ffe7c70d9b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:56.995162 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e894584-4586-4c27-9710-ffe7c70d9b5a None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:56.998985 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c033ac7-df3e-4e93-9225-42694c6b01c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:56.999594 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c033ac7-df3e-4e93-9225-42694c6b01c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:57.000174 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c033ac7-df3e-4e93-9225-42694c6b01c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:57.000174 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c033ac7-df3e-4e93-9225-42694c6b01c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:57.001974 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3c033ac7-df3e-4e93-9225-42694c6b01c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:57.012977 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c033ac7-df3e-4e93-9225-42694c6b01c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:57.014506 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c033ac7-df3e-4e93-9225-42694c6b01c2 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:57.025445 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1336/2657] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:48:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:48:57.470719 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56af633-20fc-4084-b016-bac8a5d95805 service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:57.543515 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56af633-20fc-4084-b016-bac8a5d95805 service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:57.549010 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba7470-a42e-484f-b829-7d28be234321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:57.549775 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba7470-a42e-484f-b829-7d28be234321 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:57.550103 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba7470-a42e-484f-b829-7d28be234321 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:57.550379 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba7470-a42e-484f-b829-7d28be234321 admin admin] PATH_INFO: `/v3/users/c0412b876536412f80b86653f19d1c47` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:57.551537 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ba7470-a42e-484f-b829-7d28be234321 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0412b876536412f80b86653f19d1c47)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:57.553675 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ba7470-a42e-484f-b829-7d28be234321 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:57.617371 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-52ba7470-a42e-484f-b829-7d28be234321 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0412b876536412f80b86653f19d1c47'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:57.625676 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-52ba7470-a42e-484f-b829-7d28be234321 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0412b876536412f80b86653f19d1c47'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:57.626556 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-52ba7470-a42e-484f-b829-7d28be234321 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0412b876536412f80b86653f19d1c47'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:57.637503 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-52ba7470-a42e-484f-b829-7d28be234321 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0412b876536412f80b86653f19d1c47'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:57.638523 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-52ba7470-a42e-484f-b829-7d28be234321 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0412b876536412f80b86653f19d1c47'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:57.663136 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1322/2658] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:48:57 2022] DELETE /identity/v3/users/c0412b876536412f80b86653f19d1c47 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:57.667697 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17620ba7-a842-41b0-b71c-80070a19051d None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:57.668051 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17620ba7-a842-41b0-b71c-80070a19051d None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:57.668388 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17620ba7-a842-41b0-b71c-80070a19051d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:57.693480 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-17620ba7-a842-41b0-b71c-80070a19051d None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:57.694841 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-17620ba7-a842-41b0-b71c-80070a19051d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:57.885934 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1337/2659] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:48:57.897115 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba7470-a42e-484f-b829-7d28be234321 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:57.899222 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-52ba7470-a42e-484f-b829-7d28be234321 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:58.002115 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:58.011994 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba7470-a42e-484f-b829-7d28be234321 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:58.025154 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4895c5-ef89-4e15-950c-865199b8afc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:58.025154 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4895c5-ef89-4e15-950c-865199b8afc8 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:58.025154 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4895c5-ef89-4e15-950c-865199b8afc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:58.025615 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4895c5-ef89-4e15-950c-865199b8afc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:58.040459 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4895c5-ef89-4e15-950c-865199b8afc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:58.043751 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4895c5-ef89-4e15-950c-865199b8afc8 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:58.060223 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1323/2660] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:48:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:48:58.453292 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17620ba7-a842-41b0-b71c-80070a19051d None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:58.455328 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-17620ba7-a842-41b0-b71c-80070a19051d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:58.488389 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17620ba7-a842-41b0-b71c-80070a19051d None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:58.494526 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bae56c3-876b-4435-acaf-32a4cff874f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:58.495163 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae56c3-876b-4435-acaf-32a4cff874f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:58.498229 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae56c3-876b-4435-acaf-32a4cff874f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:58.498727 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae56c3-876b-4435-acaf-32a4cff874f8 admin admin] PATH_INFO: `/v3/projects/483eae90fd2c4c40b21436b6d2c969f8` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:58.500456 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bae56c3-876b-4435-acaf-32a4cff874f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=483eae90fd2c4c40b21436b6d2c969f8)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:58.503699 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bae56c3-876b-4435-acaf-32a4cff874f8 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:58.595640 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4bae56c3-876b-4435-acaf-32a4cff874f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '483eae90fd2c4c40b21436b6d2c969f8', 'request_id': 'req-4bae56c3-876b-4435-acaf-32a4cff874f8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:58.602747 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4bae56c3-876b-4435-acaf-32a4cff874f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '483eae90fd2c4c40b21436b6d2c969f8', 'request_id': 'req-4bae56c3-876b-4435-acaf-32a4cff874f8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:58.615206 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4bae56c3-876b-4435-acaf-32a4cff874f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '483eae90fd2c4c40b21436b6d2c969f8', 'request_id': 'req-4bae56c3-876b-4435-acaf-32a4cff874f8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:58.618142 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:48:58.633363 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4bae56c3-876b-4435-acaf-32a4cff874f8 admin admin] The token cache is being invalidate because project 483eae90fd2c4c40b21436b6d2c969f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:48:58.634063 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4bae56c3-876b-4435-acaf-32a4cff874f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:58.635023 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4bae56c3-876b-4435-acaf-32a4cff874f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:58.636926 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1338/2661] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:48:58 2022] DELETE /identity/v3/projects/483eae90fd2c4c40b21436b6d2c969f8 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:58.640946 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4895c5-ef89-4e15-950c-865199b8afc8 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:58.652593 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4895c5-ef89-4e15-950c-865199b8afc8 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:58.764121 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ed9d9-59ea-46e2-ba81-c5fc68f5e5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:58.764121 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ed9d9-59ea-46e2-ba81-c5fc68f5e5a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:58.764121 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ed9d9-59ea-46e2-ba81-c5fc68f5e5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:58.764578 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ed9d9-59ea-46e2-ba81-c5fc68f5e5a7 admin admin] PATH_INFO: `/v3/users/fe8da8fd3da34b9ab9a4480ec71b471f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:58.765498 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092ed9d9-59ea-46e2-ba81-c5fc68f5e5a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe8da8fd3da34b9ab9a4480ec71b471f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:58.767615 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092ed9d9-59ea-46e2-ba81-c5fc68f5e5a7 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:58.871710 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-092ed9d9-59ea-46e2-ba81-c5fc68f5e5a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe8da8fd3da34b9ab9a4480ec71b471f'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:58.886605 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-092ed9d9-59ea-46e2-ba81-c5fc68f5e5a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe8da8fd3da34b9ab9a4480ec71b471f'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:58.887861 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-092ed9d9-59ea-46e2-ba81-c5fc68f5e5a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe8da8fd3da34b9ab9a4480ec71b471f'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:58.896746 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-092ed9d9-59ea-46e2-ba81-c5fc68f5e5a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe8da8fd3da34b9ab9a4480ec71b471f'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:58.897575 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-092ed9d9-59ea-46e2-ba81-c5fc68f5e5a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe8da8fd3da34b9ab9a4480ec71b471f'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:58.937557 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1324/2662] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:48:58 2022] DELETE /identity/v3/users/fe8da8fd3da34b9ab9a4480ec71b471f => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:48:59.574115 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bae56c3-876b-4435-acaf-32a4cff874f8 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:48:59.575760 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4bae56c3-876b-4435-acaf-32a4cff874f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:59.600686 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bae56c3-876b-4435-acaf-32a4cff874f8 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:48:59.730738 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc03eddd-0eea-4803-a029-6119af440e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:48:59.731719 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03eddd-0eea-4803-a029-6119af440e0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:59.732157 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03eddd-0eea-4803-a029-6119af440e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:59.732758 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03eddd-0eea-4803-a029-6119af440e0d admin admin] PATH_INFO: `/v3/projects/c05c60ec4e9a4293aa90bca3e4e614c6` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:59.733969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc03eddd-0eea-4803-a029-6119af440e0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c05c60ec4e9a4293aa90bca3e4e614c6)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:48:59.735916 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc03eddd-0eea-4803-a029-6119af440e0d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:48:59.806434 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-bc03eddd-0eea-4803-a029-6119af440e0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c05c60ec4e9a4293aa90bca3e4e614c6', 'request_id': 'req-bc03eddd-0eea-4803-a029-6119af440e0d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:59.814579 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-bc03eddd-0eea-4803-a029-6119af440e0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c05c60ec4e9a4293aa90bca3e4e614c6', 'request_id': 'req-bc03eddd-0eea-4803-a029-6119af440e0d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:59.827735 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-bc03eddd-0eea-4803-a029-6119af440e0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c05c60ec4e9a4293aa90bca3e4e614c6', 'request_id': 'req-bc03eddd-0eea-4803-a029-6119af440e0d'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:59.842437 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-bc03eddd-0eea-4803-a029-6119af440e0d admin admin] The token cache is being invalidate because project c05c60ec4e9a4293aa90bca3e4e614c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:48:59.842736 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-bc03eddd-0eea-4803-a029-6119af440e0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:59.843379 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-bc03eddd-0eea-4803-a029-6119af440e0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:48:59.845430 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1339/2663] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:48:59 2022] DELETE /identity/v3/projects/c05c60ec4e9a4293aa90bca3e4e614c6 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:48:59.852825 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ba8bf-dcef-4900-bc1d-799745285ed8 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:59.853147 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ba8bf-dcef-4900-bc1d-799745285ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:59.853452 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ba8bf-dcef-4900-bc1d-799745285ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:59.876702 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-7c3ba8bf-dcef-4900-bc1d-799745285ed8 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:59.877967 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7c3ba8bf-dcef-4900-bc1d-799745285ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:59.944665 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1325/2664] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:59.952798 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0384d4-ba76-42f2-aae3-5b0d8b0228f3 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:48:59.953782 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0384d4-ba76-42f2-aae3-5b0d8b0228f3 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:48:59.954098 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0384d4-ba76-42f2-aae3-5b0d8b0228f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:48:59.978756 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-0f0384d4-ba76-42f2-aae3-5b0d8b0228f3 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:48:59.980092 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0f0384d4-ba76-42f2-aae3-5b0d8b0228f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:48:59.990141 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1340/2665] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:48:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:48:59.994571 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3ba8bf-dcef-4900-bc1d-799745285ed8 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:00.008161 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3ba8bf-dcef-4900-bc1d-799745285ed8 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:00.021443 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7307820e-423f-4d1f-ad3d-3f87337eb3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:00.022119 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307820e-423f-4d1f-ad3d-3f87337eb3b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:00.022396 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307820e-423f-4d1f-ad3d-3f87337eb3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:00.022667 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307820e-423f-4d1f-ad3d-3f87337eb3b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:00.023517 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7307820e-423f-4d1f-ad3d-3f87337eb3b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:49:00.023864 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7307820e-423f-4d1f-ad3d-3f87337eb3b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:00.025486 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7307820e-423f-4d1f-ad3d-3f87337eb3b1 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:00.028850 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:00.041251 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1326/2666] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:48:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:49:00.046245 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0384d4-ba76-42f2-aae3-5b0d8b0228f3 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:00.070202 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0384d4-ba76-42f2-aae3-5b0d8b0228f3 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:00.075710 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f213c4-9b26-42d3-9e61-f87e0c65c4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:00.076435 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f213c4-9b26-42d3-9e61-f87e0c65c4e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:00.076721 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f213c4-9b26-42d3-9e61-f87e0c65c4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:00.077383 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f213c4-9b26-42d3-9e61-f87e0c65c4e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:00.078462 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f213c4-9b26-42d3-9e61-f87e0c65c4e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:00.081069 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f213c4-9b26-42d3-9e61-f87e0c65c4e9 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:00.116553 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1341/2667] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:49:00 2022] POST /identity/v3/projects => generated 373 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:49:00.117746 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7307820e-423f-4d1f-ad3d-3f87337eb3b1 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:00.138090 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7307820e-423f-4d1f-ad3d-3f87337eb3b1 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:00.195470 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba9dce-afb4-4e52-8e7c-e7b51a4893e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:00.196084 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba9dce-afb4-4e52-8e7c-e7b51a4893e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:00.196369 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba9dce-afb4-4e52-8e7c-e7b51a4893e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:00.197052 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba9dce-afb4-4e52-8e7c-e7b51a4893e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:00.197346 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eba9dce-afb4-4e52-8e7c-e7b51a4893e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:00.199364 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eba9dce-afb4-4e52-8e7c-e7b51a4893e7 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:00.222029 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1327/2668] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:49:00 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:49:00.228772 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f213c4-9b26-42d3-9e61-f87e0c65c4e9 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:00.245030 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f213c4-9b26-42d3-9e61-f87e0c65c4e9 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:00.250406 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6d9797-7f7a-4e65-82c4-a4ad852e5d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:00.251460 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d9797-7f7a-4e65-82c4-a4ad852e5d64 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:00.252180 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d9797-7f7a-4e65-82c4-a4ad852e5d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:00.253283 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d9797-7f7a-4e65-82c4-a4ad852e5d64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:00.254997 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6d9797-7f7a-4e65-82c4-a4ad852e5d64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:00.257829 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6d9797-7f7a-4e65-82c4-a4ad852e5d64 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:00.276729 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1342/2669] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:49:00 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:49:00.276947 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba9dce-afb4-4e52-8e7c-e7b51a4893e7 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:00.287329 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba9dce-afb4-4e52-8e7c-e7b51a4893e7 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:00.292529 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090bd928-1ebe-42ba-b364-3f4435e335eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:00.293139 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090bd928-1ebe-42ba-b364-3f4435e335eb admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:00.293411 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090bd928-1ebe-42ba-b364-3f4435e335eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:00.293673 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090bd928-1ebe-42ba-b364-3f4435e335eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:00.294361 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090bd928-1ebe-42ba-b364-3f4435e335eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:00.295926 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090bd928-1ebe-42ba-b364-3f4435e335eb admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:00.308430 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1328/2670] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:49:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:49:00.312526 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6d9797-7f7a-4e65-82c4-a4ad852e5d64 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:00.323054 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6d9797-7f7a-4e65-82c4-a4ad852e5d64 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:00.327665 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a39242-e031-4371-952a-34969d6e539b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:00.328315 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a39242-e031-4371-952a-34969d6e539b admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:00.328605 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a39242-e031-4371-952a-34969d6e539b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:00.328881 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a39242-e031-4371-952a-34969d6e539b admin admin] PATH_INFO: `/v3/projects/ae266e79de9b4ea79387b6f45db2af92/users/e45135ddeddc41d6a82a62fba8aa5e1e/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:00.343581 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a39242-e031-4371-952a-34969d6e539b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae266e79de9b4ea79387b6f45db2af92, user_id=e45135ddeddc41d6a82a62fba8aa5e1e, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:00.346247 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a39242-e031-4371-952a-34969d6e539b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:00.360293 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1343/2671] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:49:00 2022] PUT /identity/v3/projects/ae266e79de9b4ea79387b6f45db2af92/users/e45135ddeddc41d6a82a62fba8aa5e1e/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:49:00.371371 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17eda04-215a-4cf1-bfb4-235b93b6d9ab None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:00.371652 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17eda04-215a-4cf1-bfb4-235b93b6d9ab None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:00.371918 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17eda04-215a-4cf1-bfb4-235b93b6d9ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:00.400715 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-d17eda04-215a-4cf1-bfb4-235b93b6d9ab None None] MFA Rules not processed for user `e45135ddeddc41d6a82a62fba8aa5e1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:00.402053 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d17eda04-215a-4cf1-bfb4-235b93b6d9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:00.480419 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1329/2672] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:49:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:49:00.492241 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a39242-e031-4371-952a-34969d6e539b admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:00.494514 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a6a39242-e031-4371-952a-34969d6e539b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:00.516879 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:00.560926 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a39242-e031-4371-952a-34969d6e539b admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:00.565468 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9596ba-80e2-45a3-bab1-6b03e7b2369b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:00.566821 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9596ba-80e2-45a3-bab1-6b03e7b2369b service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:00.566821 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9596ba-80e2-45a3-bab1-6b03e7b2369b service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:00.566821 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9596ba-80e2-45a3-bab1-6b03e7b2369b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:00.578459 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9596ba-80e2-45a3-bab1-6b03e7b2369b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:00.580676 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9596ba-80e2-45a3-bab1-6b03e7b2369b service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:00.590995 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1344/2673] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:49:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:49:00.614263 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17eda04-215a-4cf1-bfb4-235b93b6d9ab None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:00.682222 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17eda04-215a-4cf1-bfb4-235b93b6d9ab None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:00.686767 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e48ac7-9486-48f1-a2bb-f33103b3db73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:00.687426 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e48ac7-9486-48f1-a2bb-f33103b3db73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:00.687742 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e48ac7-9486-48f1-a2bb-f33103b3db73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:00.688068 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e48ac7-9486-48f1-a2bb-f33103b3db73 admin admin] PATH_INFO: `/v3/users/e45135ddeddc41d6a82a62fba8aa5e1e` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:00.689215 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e48ac7-9486-48f1-a2bb-f33103b3db73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e45135ddeddc41d6a82a62fba8aa5e1e)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:00.691156 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e48ac7-9486-48f1-a2bb-f33103b3db73 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:00.696560 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:00.746402 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-74e48ac7-9486-48f1-a2bb-f33103b3db73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e45135ddeddc41d6a82a62fba8aa5e1e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:00.755610 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-74e48ac7-9486-48f1-a2bb-f33103b3db73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e45135ddeddc41d6a82a62fba8aa5e1e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:00.756319 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-74e48ac7-9486-48f1-a2bb-f33103b3db73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e45135ddeddc41d6a82a62fba8aa5e1e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:00.767664 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-74e48ac7-9486-48f1-a2bb-f33103b3db73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e45135ddeddc41d6a82a62fba8aa5e1e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:00.768286 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-74e48ac7-9486-48f1-a2bb-f33103b3db73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e45135ddeddc41d6a82a62fba8aa5e1e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:00.793256 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1330/2674] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:49:00 2022] DELETE /identity/v3/users/e45135ddeddc41d6a82a62fba8aa5e1e => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:49:00.795058 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef4fde-0d2e-4867-8e31-0f45e5d29481 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:00.795216 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef4fde-0d2e-4867-8e31-0f45e5d29481 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:00.795466 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef4fde-0d2e-4867-8e31-0f45e5d29481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:00.818443 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-e7ef4fde-0d2e-4867-8e31-0f45e5d29481 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:00.819239 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e7ef4fde-0d2e-4867-8e31-0f45e5d29481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:00.849198 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e48ac7-9486-48f1-a2bb-f33103b3db73 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:00.850638 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-74e48ac7-9486-48f1-a2bb-f33103b3db73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:00.883682 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1345/2675] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:49:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:49:00.893426 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef4fde-0d2e-4867-8e31-0f45e5d29481 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:00.894976 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e7ef4fde-0d2e-4867-8e31-0f45e5d29481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:00.924767 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e48ac7-9486-48f1-a2bb-f33103b3db73 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:00.932097 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8e8b5b-8c7f-4add-9dbc-bc65c63b6a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:00.932823 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e8b5b-8c7f-4add-9dbc-bc65c63b6a17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:00.933105 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e8b5b-8c7f-4add-9dbc-bc65c63b6a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:00.933451 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e8b5b-8c7f-4add-9dbc-bc65c63b6a17 admin admin] PATH_INFO: `/v3/users/1b4e022e090a4979a54e4de5cb44bc86` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:00.935056 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8e8b5b-8c7f-4add-9dbc-bc65c63b6a17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b4e022e090a4979a54e4de5cb44bc86)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:00.937116 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8e8b5b-8c7f-4add-9dbc-bc65c63b6a17 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:00.975193 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef4fde-0d2e-4867-8e31-0f45e5d29481 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:00.987215 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c17a7-ee33-4797-81fa-b6d23b933e89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:00.988110 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c17a7-ee33-4797-81fa-b6d23b933e89 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:00.988905 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c17a7-ee33-4797-81fa-b6d23b933e89 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:00.989259 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c17a7-ee33-4797-81fa-b6d23b933e89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:00.999652 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8c17a7-ee33-4797-81fa-b6d23b933e89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:01.001127 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8c17a7-ee33-4797-81fa-b6d23b933e89 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:01.005330 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-8e8e8b5b-8c7f-4add-9dbc-bc65c63b6a17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b4e022e090a4979a54e4de5cb44bc86'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:01.011976 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1346/2676] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:49:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:49:01.016697 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-8e8e8b5b-8c7f-4add-9dbc-bc65c63b6a17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b4e022e090a4979a54e4de5cb44bc86'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:01.017542 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-8e8e8b5b-8c7f-4add-9dbc-bc65c63b6a17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b4e022e090a4979a54e4de5cb44bc86'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:01.027864 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-8e8e8b5b-8c7f-4add-9dbc-bc65c63b6a17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b4e022e090a4979a54e4de5cb44bc86'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:01.028663 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-8e8e8b5b-8c7f-4add-9dbc-bc65c63b6a17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b4e022e090a4979a54e4de5cb44bc86'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:01.051228 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1331/2677] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:49:00 2022] DELETE /identity/v3/users/1b4e022e090a4979a54e4de5cb44bc86 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:49:01.055587 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a1e5c-d989-4f1d-a22e-775f60e503c0 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:01.055898 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a1e5c-d989-4f1d-a22e-775f60e503c0 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:01.056201 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a1e5c-d989-4f1d-a22e-775f60e503c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:01.084260 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-952a1e5c-d989-4f1d-a22e-775f60e503c0 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:01.090204 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-952a1e5c-d989-4f1d-a22e-775f60e503c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:01.179856 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1347/2678] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:49:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:49:01.221261 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8e8b5b-8c7f-4add-9dbc-bc65c63b6a17 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:01.221261 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-8e8e8b5b-8c7f-4add-9dbc-bc65c63b6a17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:01.227564 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:01.348906 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8e8b5b-8c7f-4add-9dbc-bc65c63b6a17 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:01.356978 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a6228-1972-40ad-9d48-22d4e48e90c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:01.357735 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a6228-1972-40ad-9d48-22d4e48e90c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:01.358122 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a6228-1972-40ad-9d48-22d4e48e90c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:01.358466 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a6228-1972-40ad-9d48-22d4e48e90c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:01.371519 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284a6228-1972-40ad-9d48-22d4e48e90c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:01.373327 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284a6228-1972-40ad-9d48-22d4e48e90c8 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:01.384780 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952a1e5c-d989-4f1d-a22e-775f60e503c0 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:01.386906 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-952a1e5c-d989-4f1d-a22e-775f60e503c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:01.406458 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1332/2679] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:49:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:49:01.408698 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:01.421340 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952a1e5c-d989-4f1d-a22e-775f60e503c0 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:01.430233 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b6e580-0733-4dda-b88d-1cb511ed449a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:01.434008 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b6e580-0733-4dda-b88d-1cb511ed449a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:01.434639 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b6e580-0733-4dda-b88d-1cb511ed449a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:01.435388 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b6e580-0733-4dda-b88d-1cb511ed449a admin admin] PATH_INFO: `/v3/projects/ae266e79de9b4ea79387b6f45db2af92` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:01.438439 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b6e580-0733-4dda-b88d-1cb511ed449a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae266e79de9b4ea79387b6f45db2af92)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:01.440640 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b6e580-0733-4dda-b88d-1cb511ed449a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:01.520844 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c9b6e580-0733-4dda-b88d-1cb511ed449a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae266e79de9b4ea79387b6f45db2af92', 'request_id': 'req-c9b6e580-0733-4dda-b88d-1cb511ed449a'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:01.528122 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c9b6e580-0733-4dda-b88d-1cb511ed449a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae266e79de9b4ea79387b6f45db2af92', 'request_id': 'req-c9b6e580-0733-4dda-b88d-1cb511ed449a'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:01.538180 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c9b6e580-0733-4dda-b88d-1cb511ed449a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae266e79de9b4ea79387b6f45db2af92', 'request_id': 'req-c9b6e580-0733-4dda-b88d-1cb511ed449a'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:01.550476 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c9b6e580-0733-4dda-b88d-1cb511ed449a admin admin] The token cache is being invalidate because project ae266e79de9b4ea79387b6f45db2af92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:49:01.550797 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c9b6e580-0733-4dda-b88d-1cb511ed449a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:01.551436 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c9b6e580-0733-4dda-b88d-1cb511ed449a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:01.552920 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1348/2680] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:49:01 2022] DELETE /identity/v3/projects/ae266e79de9b4ea79387b6f45db2af92 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:49:01.558589 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e6770-c811-4009-8c77-86f3960aef63 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:01.559045 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e6770-c811-4009-8c77-86f3960aef63 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:01.559441 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e6770-c811-4009-8c77-86f3960aef63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:01.583267 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-179e6770-c811-4009-8c77-86f3960aef63 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:01.584498 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-179e6770-c811-4009-8c77-86f3960aef63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:01.660600 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1333/2681] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:49:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:49:01.669513 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484f77f-1551-4611-b46f-b2029ba02171 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:01.670074 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484f77f-1551-4611-b46f-b2029ba02171 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:01.670161 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484f77f-1551-4611-b46f-b2029ba02171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:01.690759 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-a484f77f-1551-4611-b46f-b2029ba02171 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:01.692569 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a484f77f-1551-4611-b46f-b2029ba02171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:01.700556 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1349/2682] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:49:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:49:01.704838 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179e6770-c811-4009-8c77-86f3960aef63 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:01.714489 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179e6770-c811-4009-8c77-86f3960aef63 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:01.719330 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276fa4d5-f92f-4023-ae5d-4b70f63bbef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:01.719941 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276fa4d5-f92f-4023-ae5d-4b70f63bbef6 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:01.720214 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276fa4d5-f92f-4023-ae5d-4b70f63bbef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:01.720478 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276fa4d5-f92f-4023-ae5d-4b70f63bbef6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:01.721118 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276fa4d5-f92f-4023-ae5d-4b70f63bbef6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:49:01.721412 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276fa4d5-f92f-4023-ae5d-4b70f63bbef6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:01.723076 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276fa4d5-f92f-4023-ae5d-4b70f63bbef6 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:01.740170 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1334/2683] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:49:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:49:01.741342 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a484f77f-1551-4611-b46f-b2029ba02171 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:01.750924 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a484f77f-1551-4611-b46f-b2029ba02171 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:01.756446 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa22700-2364-4f62-b279-b42292dcac1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:01.757048 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa22700-2364-4f62-b279-b42292dcac1b admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:01.757324 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa22700-2364-4f62-b279-b42292dcac1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:01.757592 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa22700-2364-4f62-b279-b42292dcac1b admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:01.758239 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa22700-2364-4f62-b279-b42292dcac1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:01.760331 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa22700-2364-4f62-b279-b42292dcac1b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:01.798311 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1350/2684] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:49:01 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:49:01.799437 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276fa4d5-f92f-4023-ae5d-4b70f63bbef6 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:01.810339 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276fa4d5-f92f-4023-ae5d-4b70f63bbef6 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:01.887413 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a8375c-99f0-4e34-a275-6588f1b4a24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:01.888216 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a8375c-99f0-4e34-a275-6588f1b4a24a admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:01.888491 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a8375c-99f0-4e34-a275-6588f1b4a24a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:01.888767 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a8375c-99f0-4e34-a275-6588f1b4a24a admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:01.889410 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a8375c-99f0-4e34-a275-6588f1b4a24a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:01.891375 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a8375c-99f0-4e34-a275-6588f1b4a24a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:01.916797 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa22700-2364-4f62-b279-b42292dcac1b admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:01.918679 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0fa22700-2364-4f62-b279-b42292dcac1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:01.924266 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1335/2685] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:49:01 2022] POST /identity/v3/users => generated 287 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:49:01.924913 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a8375c-99f0-4e34-a275-6588f1b4a24a admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:01.936969 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a8375c-99f0-4e34-a275-6588f1b4a24a admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:01.939393 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa22700-2364-4f62-b279-b42292dcac1b admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:01.944050 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040956c5-d468-45ff-92e4-dbf6df0a237a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:01.945021 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040956c5-d468-45ff-92e4-dbf6df0a237a admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:01.945381 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040956c5-d468-45ff-92e4-dbf6df0a237a admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:01.945729 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040956c5-d468-45ff-92e4-dbf6df0a237a admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:01.946237 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea6761-ab2a-4a63-b0e1-7f4312605558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:01.946685 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040956c5-d468-45ff-92e4-dbf6df0a237a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:01.947136 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea6761-ab2a-4a63-b0e1-7f4312605558 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:01.947480 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea6761-ab2a-4a63-b0e1-7f4312605558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:01.947819 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea6761-ab2a-4a63-b0e1-7f4312605558 admin admin] PATH_INFO: `/v3/projects/3e09ad6c880d43ba92ffe0d12699d7e4` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:01.948543 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040956c5-d468-45ff-92e4-dbf6df0a237a admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:01.949096 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea6761-ab2a-4a63-b0e1-7f4312605558 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e09ad6c880d43ba92ffe0d12699d7e4)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:01.951228 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea6761-ab2a-4a63-b0e1-7f4312605558 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:01.962186 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1336/2686] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:49:01 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:49:01.966775 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040956c5-d468-45ff-92e4-dbf6df0a237a admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:01.979456 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040956c5-d468-45ff-92e4-dbf6df0a237a admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:01.984824 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9f6be-115b-417f-be5c-36d8a87da287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:01.985986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9f6be-115b-417f-be5c-36d8a87da287 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:01.985986 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9f6be-115b-417f-be5c-36d8a87da287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:01.986217 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9f6be-115b-417f-be5c-36d8a87da287 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:01.986724 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b9f6be-115b-417f-be5c-36d8a87da287 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:01.988960 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b9f6be-115b-417f-be5c-36d8a87da287 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:01.995894 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:02.005080 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1337/2687] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:49:01 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:49:02.006045 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9f6be-115b-417f-be5c-36d8a87da287 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:02.007143 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-95ea6761-ab2a-4a63-b0e1-7f4312605558 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e09ad6c880d43ba92ffe0d12699d7e4', 'request_id': 'req-95ea6761-ab2a-4a63-b0e1-7f4312605558'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:02.013551 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-95ea6761-ab2a-4a63-b0e1-7f4312605558 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e09ad6c880d43ba92ffe0d12699d7e4', 'request_id': 'req-95ea6761-ab2a-4a63-b0e1-7f4312605558'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:02.020284 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9f6be-115b-417f-be5c-36d8a87da287 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:02.022706 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-95ea6761-ab2a-4a63-b0e1-7f4312605558 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e09ad6c880d43ba92ffe0d12699d7e4', 'request_id': 'req-95ea6761-ab2a-4a63-b0e1-7f4312605558'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:02.038557 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-95ea6761-ab2a-4a63-b0e1-7f4312605558 admin admin] The token cache is being invalidate because project 3e09ad6c880d43ba92ffe0d12699d7e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:49:02.039180 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-95ea6761-ab2a-4a63-b0e1-7f4312605558 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:02.040047 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-95ea6761-ab2a-4a63-b0e1-7f4312605558 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:02.041795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1351/2688] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:49:01 2022] DELETE /identity/v3/projects/3e09ad6c880d43ba92ffe0d12699d7e4 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:49:02.045737 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea6761-ab2a-4a63-b0e1-7f4312605558 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:02.047114 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-95ea6761-ab2a-4a63-b0e1-7f4312605558 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:02.090633 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc5b8a4-f07e-4335-bdf7-91aa1b6c2c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:02.091241 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5b8a4-f07e-4335-bdf7-91aa1b6c2c48 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:02.091515 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5b8a4-f07e-4335-bdf7-91aa1b6c2c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:02.091949 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5b8a4-f07e-4335-bdf7-91aa1b6c2c48 admin admin] PATH_INFO: `/v3/projects/8ff3f4275a4241119e7d42d93cedb907/users/83fef35a221e4ca089336d624e794247/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:02.095044 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:02.108212 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc5b8a4-f07e-4335-bdf7-91aa1b6c2c48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ff3f4275a4241119e7d42d93cedb907, user_id=83fef35a221e4ca089336d624e794247, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:02.111070 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc5b8a4-f07e-4335-bdf7-91aa1b6c2c48 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:02.117383 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea6761-ab2a-4a63-b0e1-7f4312605558 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:02.122276 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8218ae-2cf1-4feb-9a8a-0ce6c2cfdcff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:02.122903 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8218ae-2cf1-4feb-9a8a-0ce6c2cfdcff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:02.123181 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8218ae-2cf1-4feb-9a8a-0ce6c2cfdcff admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:02.123454 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8218ae-2cf1-4feb-9a8a-0ce6c2cfdcff admin admin] PATH_INFO: `/v3/users/af14b215a46c4176a693f77b8f70da4e` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:02.124154 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1338/2689] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:49:02 2022] PUT /identity/v3/projects/8ff3f4275a4241119e7d42d93cedb907/users/83fef35a221e4ca089336d624e794247/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:49:02.124571 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8218ae-2cf1-4feb-9a8a-0ce6c2cfdcff admin admin] RBAC: Authorizing `identity:delete_user(user_id=af14b215a46c4176a693f77b8f70da4e)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:02.126541 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8218ae-2cf1-4feb-9a8a-0ce6c2cfdcff admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:02.134885 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f783b87-29ff-4dac-af75-8a6ef61e9aad None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:02.135165 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f783b87-29ff-4dac-af75-8a6ef61e9aad None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:02.135435 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f783b87-29ff-4dac-af75-8a6ef61e9aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:02.165087 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-3f783b87-29ff-4dac-af75-8a6ef61e9aad None None] MFA Rules not processed for user `83fef35a221e4ca089336d624e794247`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:02.166334 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3f783b87-29ff-4dac-af75-8a6ef61e9aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:02.181378 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2f8218ae-2cf1-4feb-9a8a-0ce6c2cfdcff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af14b215a46c4176a693f77b8f70da4e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:02.182067 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2f8218ae-2cf1-4feb-9a8a-0ce6c2cfdcff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af14b215a46c4176a693f77b8f70da4e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:02.195366 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2f8218ae-2cf1-4feb-9a8a-0ce6c2cfdcff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af14b215a46c4176a693f77b8f70da4e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:02.203357 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2f8218ae-2cf1-4feb-9a8a-0ce6c2cfdcff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af14b215a46c4176a693f77b8f70da4e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:02.212095 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2f8218ae-2cf1-4feb-9a8a-0ce6c2cfdcff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af14b215a46c4176a693f77b8f70da4e'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:02.218444 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1352/2690] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:49:02 2022] DELETE /identity/v3/users/af14b215a46c4176a693f77b8f70da4e => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:49:02.238256 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1339/2691] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:49:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:49:02.250685 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8218ae-2cf1-4feb-9a8a-0ce6c2cfdcff admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:02.251492 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2f8218ae-2cf1-4feb-9a8a-0ce6c2cfdcff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:02.332498 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8218ae-2cf1-4feb-9a8a-0ce6c2cfdcff admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:02.336399 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f0f48-e082-4629-b4fb-c17731ed10b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:02.337126 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f0f48-e082-4629-b4fb-c17731ed10b8 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:02.337411 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f0f48-e082-4629-b4fb-c17731ed10b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:02.337883 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f0f48-e082-4629-b4fb-c17731ed10b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:02.407385 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f0f48-e082-4629-b4fb-c17731ed10b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:02.409176 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f0f48-e082-4629-b4fb-c17731ed10b8 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:02.420058 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1353/2692] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:49:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:49:02.529009 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f783b87-29ff-4dac-af75-8a6ef61e9aad None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:02.530648 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3f783b87-29ff-4dac-af75-8a6ef61e9aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:02.659681 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f783b87-29ff-4dac-af75-8a6ef61e9aad None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:02.666576 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f0f48-e082-4629-b4fb-c17731ed10b8 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:02.668024 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1a8f0f48-e082-4629-b4fb-c17731ed10b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:02.697590 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:02.699702 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:02.773204 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42815035-eadb-4284-8ad9-1cf16fa4375e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:02.773877 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42815035-eadb-4284-8ad9-1cf16fa4375e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:02.774160 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42815035-eadb-4284-8ad9-1cf16fa4375e admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:02.774497 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42815035-eadb-4284-8ad9-1cf16fa4375e admin admin] PATH_INFO: `/v3/projects/51de70265e5243d2b0d629631e1ba468` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:02.775431 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f0f48-e082-4629-b4fb-c17731ed10b8 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:02.775650 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42815035-eadb-4284-8ad9-1cf16fa4375e admin admin] RBAC: Authorizing `identity:delete_project(project_id=51de70265e5243d2b0d629631e1ba468)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:02.777637 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42815035-eadb-4284-8ad9-1cf16fa4375e admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:02.779506 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7359e1-1724-4aea-b3ff-b208ba8438aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:02.780202 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7359e1-1724-4aea-b3ff-b208ba8438aa service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:02.782256 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7359e1-1724-4aea-b3ff-b208ba8438aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:02.782768 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7359e1-1724-4aea-b3ff-b208ba8438aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:02.785532 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-fa7359e1-1724-4aea-b3ff-b208ba8438aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:02.801560 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7359e1-1724-4aea-b3ff-b208ba8438aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:02.803387 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7359e1-1724-4aea-b3ff-b208ba8438aa service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:02.821970 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1354/2693] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:49:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:49:02.848748 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-42815035-eadb-4284-8ad9-1cf16fa4375e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51de70265e5243d2b0d629631e1ba468', 'request_id': 'req-42815035-eadb-4284-8ad9-1cf16fa4375e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:02.860261 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-42815035-eadb-4284-8ad9-1cf16fa4375e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51de70265e5243d2b0d629631e1ba468', 'request_id': 'req-42815035-eadb-4284-8ad9-1cf16fa4375e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:02.869185 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-42815035-eadb-4284-8ad9-1cf16fa4375e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51de70265e5243d2b0d629631e1ba468', 'request_id': 'req-42815035-eadb-4284-8ad9-1cf16fa4375e'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:02.884144 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-42815035-eadb-4284-8ad9-1cf16fa4375e admin admin] The token cache is being invalidate because project 51de70265e5243d2b0d629631e1ba468 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:49:02.884633 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-42815035-eadb-4284-8ad9-1cf16fa4375e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:02.885483 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-42815035-eadb-4284-8ad9-1cf16fa4375e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:02.887574 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1340/2694] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:49:02 2022] DELETE /identity/v3/projects/51de70265e5243d2b0d629631e1ba468 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:49:02.891767 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7359e1-1724-4aea-b3ff-b208ba8438aa service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:02.902998 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7359e1-1724-4aea-b3ff-b208ba8438aa service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:02.990234 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14847ada-8f06-4552-bece-081cd63b5c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:02.990887 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14847ada-8f06-4552-bece-081cd63b5c98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:02.991167 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14847ada-8f06-4552-bece-081cd63b5c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:02.991518 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14847ada-8f06-4552-bece-081cd63b5c98 admin admin] PATH_INFO: `/v3/users/eb8f4cea637f4141b7eeec0f5d257158` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:02.992752 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14847ada-8f06-4552-bece-081cd63b5c98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb8f4cea637f4141b7eeec0f5d257158)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:02.994904 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14847ada-8f06-4552-bece-081cd63b5c98 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:03.043768 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-14847ada-8f06-4552-bece-081cd63b5c98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb8f4cea637f4141b7eeec0f5d257158'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:03.044520 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-14847ada-8f06-4552-bece-081cd63b5c98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb8f4cea637f4141b7eeec0f5d257158'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:03.056824 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-14847ada-8f06-4552-bece-081cd63b5c98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb8f4cea637f4141b7eeec0f5d257158'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:03.064592 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-14847ada-8f06-4552-bece-081cd63b5c98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb8f4cea637f4141b7eeec0f5d257158'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:03.073430 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-14847ada-8f06-4552-bece-081cd63b5c98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb8f4cea637f4141b7eeec0f5d257158'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:03.079711 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1355/2695] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:49:02 2022] DELETE /identity/v3/users/eb8f4cea637f4141b7eeec0f5d257158 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:49:03.358576 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42815035-eadb-4284-8ad9-1cf16fa4375e admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:03.360023 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-42815035-eadb-4284-8ad9-1cf16fa4375e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:03.373261 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42815035-eadb-4284-8ad9-1cf16fa4375e admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:03.436499 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad545011-c573-47d9-977c-d57f0ba37c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:03.437135 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad545011-c573-47d9-977c-d57f0ba37c60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:03.437413 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad545011-c573-47d9-977c-d57f0ba37c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:03.437687 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad545011-c573-47d9-977c-d57f0ba37c60 admin admin] PATH_INFO: `/v3/projects/0567ed059c4640c485c043332e33a10e` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:03.438844 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad545011-c573-47d9-977c-d57f0ba37c60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0567ed059c4640c485c043332e33a10e)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:03.442160 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad545011-c573-47d9-977c-d57f0ba37c60 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:03.491638 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ad545011-c573-47d9-977c-d57f0ba37c60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0567ed059c4640c485c043332e33a10e', 'request_id': 'req-ad545011-c573-47d9-977c-d57f0ba37c60'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:03.496158 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:03.499515 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ad545011-c573-47d9-977c-d57f0ba37c60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0567ed059c4640c485c043332e33a10e', 'request_id': 'req-ad545011-c573-47d9-977c-d57f0ba37c60'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:03.508158 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ad545011-c573-47d9-977c-d57f0ba37c60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0567ed059c4640c485c043332e33a10e', 'request_id': 'req-ad545011-c573-47d9-977c-d57f0ba37c60'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:03.518784 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ad545011-c573-47d9-977c-d57f0ba37c60 admin admin] The token cache is being invalidate because project 0567ed059c4640c485c043332e33a10e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:49:03.519389 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ad545011-c573-47d9-977c-d57f0ba37c60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:03.520031 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ad545011-c573-47d9-977c-d57f0ba37c60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:03.521545 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1341/2696] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:49:03 2022] DELETE /identity/v3/projects/0567ed059c4640c485c043332e33a10e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:49:03.526643 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c35462-2e6c-465e-9a4c-4c1780521b40 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:03.526953 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c35462-2e6c-465e-9a4c-4c1780521b40 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:03.527227 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c35462-2e6c-465e-9a4c-4c1780521b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:03.547035 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-31c35462-2e6c-465e-9a4c-4c1780521b40 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:03.548238 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-31c35462-2e6c-465e-9a4c-4c1780521b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:03.581265 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:03.610496 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1356/2697] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:49:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:49:03.619367 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cac1e6-af70-4b36-8bde-073fda3d5770 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:03.619657 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cac1e6-af70-4b36-8bde-073fda3d5770 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:03.619931 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cac1e6-af70-4b36-8bde-073fda3d5770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:03.639970 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-19cac1e6-af70-4b36-8bde-073fda3d5770 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:03.641163 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-19cac1e6-af70-4b36-8bde-073fda3d5770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:03.649726 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1342/2698] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:49:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:49:03.654019 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c35462-2e6c-465e-9a4c-4c1780521b40 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:03.664050 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c35462-2e6c-465e-9a4c-4c1780521b40 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:03.668887 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743e8d5e-cf5e-4bbc-af46-3651f404913f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:03.669491 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e8d5e-cf5e-4bbc-af46-3651f404913f admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:03.669775 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e8d5e-cf5e-4bbc-af46-3651f404913f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:03.670083 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e8d5e-cf5e-4bbc-af46-3651f404913f admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:03.670746 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743e8d5e-cf5e-4bbc-af46-3651f404913f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:49:03.671049 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743e8d5e-cf5e-4bbc-af46-3651f404913f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:03.672671 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743e8d5e-cf5e-4bbc-af46-3651f404913f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:03.685662 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1357/2699] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:49:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:49:03.689964 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cac1e6-af70-4b36-8bde-073fda3d5770 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:03.699251 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cac1e6-af70-4b36-8bde-073fda3d5770 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:03.703851 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d10d7-34c4-4e4f-867c-bc86ede55362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:03.704438 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d10d7-34c4-4e4f-867c-bc86ede55362 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:03.704712 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d10d7-34c4-4e4f-867c-bc86ede55362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:03.704974 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d10d7-34c4-4e4f-867c-bc86ede55362 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:03.705604 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76d10d7-34c4-4e4f-867c-bc86ede55362 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:03.707574 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76d10d7-34c4-4e4f-867c-bc86ede55362 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:03.741155 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1343/2700] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:49:03 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:49:03.742402 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743e8d5e-cf5e-4bbc-af46-3651f404913f admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:03.755978 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743e8d5e-cf5e-4bbc-af46-3651f404913f admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:03.831307 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3066706-9d94-43b8-aee1-b2c1d3a475e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:03.834137 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3066706-9d94-43b8-aee1-b2c1d3a475e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:03.834137 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3066706-9d94-43b8-aee1-b2c1d3a475e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:03.834137 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3066706-9d94-43b8-aee1-b2c1d3a475e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:03.834137 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3066706-9d94-43b8-aee1-b2c1d3a475e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:03.835729 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3066706-9d94-43b8-aee1-b2c1d3a475e2 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:03.869948 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1358/2701] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:49:03 2022] POST /identity/v3/users => generated 292 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:49:03.874284 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d10d7-34c4-4e4f-867c-bc86ede55362 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:03.885397 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d10d7-34c4-4e4f-867c-bc86ede55362 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:03.903793 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab64a4a-7d01-4007-9af7-e6c17be4c4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:03.904937 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab64a4a-7d01-4007-9af7-e6c17be4c4e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:03.905393 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab64a4a-7d01-4007-9af7-e6c17be4c4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:03.905993 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab64a4a-7d01-4007-9af7-e6c17be4c4e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:03.906806 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab64a4a-7d01-4007-9af7-e6c17be4c4e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:03.908569 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab64a4a-7d01-4007-9af7-e6c17be4c4e3 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:03.930238 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1344/2702] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:49:03.934343 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3066706-9d94-43b8-aee1-b2c1d3a475e2 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:03.947505 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3066706-9d94-43b8-aee1-b2c1d3a475e2 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:03.962699 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b541c4-09a6-478b-99c5-cb2872727aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:03.963171 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b541c4-09a6-478b-99c5-cb2872727aef admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:03.963445 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b541c4-09a6-478b-99c5-cb2872727aef admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:03.963720 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b541c4-09a6-478b-99c5-cb2872727aef admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:03.964556 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b541c4-09a6-478b-99c5-cb2872727aef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:03.967051 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b541c4-09a6-478b-99c5-cb2872727aef admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:03.993207 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1359/2703] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:49:03 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:49:03.997465 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab64a4a-7d01-4007-9af7-e6c17be4c4e3 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:04.008100 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab64a4a-7d01-4007-9af7-e6c17be4c4e3 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:04.013002 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49239860-dcd3-4f39-beb0-fb42880e5f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:04.013775 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49239860-dcd3-4f39-beb0-fb42880e5f77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:04.014183 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49239860-dcd3-4f39-beb0-fb42880e5f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:04.014581 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49239860-dcd3-4f39-beb0-fb42880e5f77 admin admin] PATH_INFO: `/v3/projects/069626c3525d47a9b67f1e6d81b47b2a/users/a7a959e57e0c4713a855844645fdc26a/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:04.031164 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49239860-dcd3-4f39-beb0-fb42880e5f77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=069626c3525d47a9b67f1e6d81b47b2a, user_id=a7a959e57e0c4713a855844645fdc26a, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:04.034125 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49239860-dcd3-4f39-beb0-fb42880e5f77 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:04.055414 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1345/2704] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:49:03 2022] PUT /identity/v3/projects/069626c3525d47a9b67f1e6d81b47b2a/users/a7a959e57e0c4713a855844645fdc26a/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:49:04.060530 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38a2ba-f642-46ae-848b-060ba7124241 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:04.060888 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38a2ba-f642-46ae-848b-060ba7124241 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:04.061235 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38a2ba-f642-46ae-848b-060ba7124241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:04.083747 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-4f38a2ba-f642-46ae-848b-060ba7124241 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:04.085145 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4f38a2ba-f642-46ae-848b-060ba7124241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:04.289607 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1360/2705] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:49:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:49:04.299799 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49239860-dcd3-4f39-beb0-fb42880e5f77 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:04.302335 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-49239860-dcd3-4f39-beb0-fb42880e5f77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:04.407015 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49239860-dcd3-4f39-beb0-fb42880e5f77 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:04.413339 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a163dbcf-5e76-4f99-8fc2-15472eeaaaf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:04.414254 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163dbcf-5e76-4f99-8fc2-15472eeaaaf2 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:04.414725 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163dbcf-5e76-4f99-8fc2-15472eeaaaf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:04.415137 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a163dbcf-5e76-4f99-8fc2-15472eeaaaf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:04.446054 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a163dbcf-5e76-4f99-8fc2-15472eeaaaf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:04.448172 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a163dbcf-5e76-4f99-8fc2-15472eeaaaf2 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:04.463236 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1346/2706] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:49:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:49:06.711327 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f38a2ba-f642-46ae-848b-060ba7124241 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:06.712746 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4f38a2ba-f642-46ae-848b-060ba7124241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:06.781057 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:06.791766 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a163dbcf-5e76-4f99-8fc2-15472eeaaaf2 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:06.795949 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-a163dbcf-5e76-4f99-8fc2-15472eeaaaf2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:06.816136 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a163dbcf-5e76-4f99-8fc2-15472eeaaaf2 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:06.825218 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e377edd3-be42-4500-8710-3a1f247cada3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:06.826212 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e377edd3-be42-4500-8710-3a1f247cada3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:06.826886 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e377edd3-be42-4500-8710-3a1f247cada3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:06.827695 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e377edd3-be42-4500-8710-3a1f247cada3 admin admin] PATH_INFO: `/v3/users/298dfca23e004969a0c907e18c7237a4` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:06.834209 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f38a2ba-f642-46ae-848b-060ba7124241 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:06.837711 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e377edd3-be42-4500-8710-3a1f247cada3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=298dfca23e004969a0c907e18c7237a4)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:06.841454 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e377edd3-be42-4500-8710-3a1f247cada3 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:06.845140 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd5b1d5-7aed-42b7-a7a6-f722fc08a979 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de934d0105bd4d01ba6c06a4a0d36704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:06.846022 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd5b1d5-7aed-42b7-a7a6-f722fc08a979 service placement] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:06.846389 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd5b1d5-7aed-42b7-a7a6-f722fc08a979 service placement] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:06.847063 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd5b1d5-7aed-42b7-a7a6-f722fc08a979 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:06.850165 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ecd5b1d5-7aed-42b7-a7a6-f722fc08a979 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:06.868215 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd5b1d5-7aed-42b7-a7a6-f722fc08a979 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:06.869962 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd5b1d5-7aed-42b7-a7a6-f722fc08a979 service placement] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:06.887467 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1361/2707] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:49:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:49:06.913755 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e377edd3-be42-4500-8710-3a1f247cada3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '298dfca23e004969a0c907e18c7237a4'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:06.923038 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e377edd3-be42-4500-8710-3a1f247cada3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '298dfca23e004969a0c907e18c7237a4'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:06.924161 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e377edd3-be42-4500-8710-3a1f247cada3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '298dfca23e004969a0c907e18c7237a4'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:06.941619 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e377edd3-be42-4500-8710-3a1f247cada3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '298dfca23e004969a0c907e18c7237a4'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:06.942647 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-e377edd3-be42-4500-8710-3a1f247cada3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '298dfca23e004969a0c907e18c7237a4'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:06.951492 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:06.963147 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1347/2708] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:49:06 2022] DELETE /identity/v3/users/298dfca23e004969a0c907e18c7237a4 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:49:07.217372 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd5b1d5-7aed-42b7-a7a6-f722fc08a979 service placement] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:07.240906 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd5b1d5-7aed-42b7-a7a6-f722fc08a979 service placement] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:07.326582 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6788e2-d0d1-42ce-9f6e-b7359b3ecda5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:07.327307 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6788e2-d0d1-42ce-9f6e-b7359b3ecda5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:07.327668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6788e2-d0d1-42ce-9f6e-b7359b3ecda5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:07.328228 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6788e2-d0d1-42ce-9f6e-b7359b3ecda5 admin admin] PATH_INFO: `/v3/projects/335ecfeb5c6a436e909618c1305593bb` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:07.329904 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6788e2-d0d1-42ce-9f6e-b7359b3ecda5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=335ecfeb5c6a436e909618c1305593bb)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:07.333103 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6788e2-d0d1-42ce-9f6e-b7359b3ecda5 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:07.400640 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7c6788e2-d0d1-42ce-9f6e-b7359b3ecda5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '335ecfeb5c6a436e909618c1305593bb', 'request_id': 'req-7c6788e2-d0d1-42ce-9f6e-b7359b3ecda5'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:07.408726 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7c6788e2-d0d1-42ce-9f6e-b7359b3ecda5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '335ecfeb5c6a436e909618c1305593bb', 'request_id': 'req-7c6788e2-d0d1-42ce-9f6e-b7359b3ecda5'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:07.424511 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7c6788e2-d0d1-42ce-9f6e-b7359b3ecda5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '335ecfeb5c6a436e909618c1305593bb', 'request_id': 'req-7c6788e2-d0d1-42ce-9f6e-b7359b3ecda5'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:07.438567 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7c6788e2-d0d1-42ce-9f6e-b7359b3ecda5 admin admin] The token cache is being invalidate because project 335ecfeb5c6a436e909618c1305593bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:49:07.438856 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7c6788e2-d0d1-42ce-9f6e-b7359b3ecda5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:07.439722 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7c6788e2-d0d1-42ce-9f6e-b7359b3ecda5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:07.442795 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1362/2709] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:49:07 2022] DELETE /identity/v3/projects/335ecfeb5c6a436e909618c1305593bb => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:49:07.448296 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e79e4b-3db4-4716-a184-b4e5d759877e None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:07.448601 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e79e4b-3db4-4716-a184-b4e5d759877e None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:07.448868 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e79e4b-3db4-4716-a184-b4e5d759877e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:07.476118 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-89e79e4b-3db4-4716-a184-b4e5d759877e None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:07.477341 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-89e79e4b-3db4-4716-a184-b4e5d759877e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:07.649704 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1348/2710] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:49:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:49:07.666130 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656fb140-1afc-498c-9ef6-efc56cb224d1 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:07.666130 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656fb140-1afc-498c-9ef6-efc56cb224d1 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:07.666130 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656fb140-1afc-498c-9ef6-efc56cb224d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:07.683800 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-656fb140-1afc-498c-9ef6-efc56cb224d1 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:07.685241 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-656fb140-1afc-498c-9ef6-efc56cb224d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:07.698771 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1363/2711] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:49:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:49:07.703012 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e79e4b-3db4-4716-a184-b4e5d759877e None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:07.731197 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e79e4b-3db4-4716-a184-b4e5d759877e None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:07.742788 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d2354-8e86-4836-a985-b23f2efcb57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:07.743401 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d2354-8e86-4836-a985-b23f2efcb57d admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:07.743685 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d2354-8e86-4836-a985-b23f2efcb57d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:07.743958 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d2354-8e86-4836-a985-b23f2efcb57d admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:07.744623 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132d2354-8e86-4836-a985-b23f2efcb57d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:49:07.745120 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132d2354-8e86-4836-a985-b23f2efcb57d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:07.751640 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132d2354-8e86-4836-a985-b23f2efcb57d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:07.781522 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656fb140-1afc-498c-9ef6-efc56cb224d1 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:07.783462 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1349/2712] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:49:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:49:07.801526 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656fb140-1afc-498c-9ef6-efc56cb224d1 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:07.815437 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f3754-7e9a-4d39-a046-636f4c69a27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:07.817254 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f3754-7e9a-4d39-a046-636f4c69a27d admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:07.819645 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f3754-7e9a-4d39-a046-636f4c69a27d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:07.820114 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f3754-7e9a-4d39-a046-636f4c69a27d admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:07.821061 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88f3754-7e9a-4d39-a046-636f4c69a27d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:07.824764 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88f3754-7e9a-4d39-a046-636f4c69a27d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:07.899977 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d2354-8e86-4836-a985-b23f2efcb57d admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:07.905392 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1364/2713] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:49:07 2022] POST /identity/v3/projects => generated 373 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:49:07.911477 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d2354-8e86-4836-a985-b23f2efcb57d admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:08.054314 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7cebf1-398c-4ae8-ad85-562bc851cb07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:08.054860 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7cebf1-398c-4ae8-ad85-562bc851cb07 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:08.055137 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7cebf1-398c-4ae8-ad85-562bc851cb07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:08.055406 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7cebf1-398c-4ae8-ad85-562bc851cb07 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:08.056076 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7cebf1-398c-4ae8-ad85-562bc851cb07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:08.058070 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7cebf1-398c-4ae8-ad85-562bc851cb07 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:08.106049 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1350/2714] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:49:07 2022] POST /identity/v3/users => generated 287 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:49:08.107561 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f3754-7e9a-4d39-a046-636f4c69a27d admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:08.125268 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f3754-7e9a-4d39-a046-636f4c69a27d admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:08.138497 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cea22a-604f-4b00-8770-d51345f3cb38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:08.139123 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cea22a-604f-4b00-8770-d51345f3cb38 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:08.140278 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cea22a-604f-4b00-8770-d51345f3cb38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:08.140557 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cea22a-604f-4b00-8770-d51345f3cb38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:08.141209 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cea22a-604f-4b00-8770-d51345f3cb38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:08.142862 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cea22a-604f-4b00-8770-d51345f3cb38 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:08.172353 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1365/2715] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:49:08 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:49:08.172566 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7cebf1-398c-4ae8-ad85-562bc851cb07 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:08.190328 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7cebf1-398c-4ae8-ad85-562bc851cb07 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:08.203298 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cbc7b5-bf55-437c-9896-28105688f96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:08.203923 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cbc7b5-bf55-437c-9896-28105688f96c admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:08.204201 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cbc7b5-bf55-437c-9896-28105688f96c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:08.204471 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cbc7b5-bf55-437c-9896-28105688f96c admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:08.205122 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cbc7b5-bf55-437c-9896-28105688f96c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:08.207316 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cbc7b5-bf55-437c-9896-28105688f96c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:08.258249 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1351/2716] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:49:08 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:49:08.262549 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cea22a-604f-4b00-8770-d51345f3cb38 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:08.275848 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cea22a-604f-4b00-8770-d51345f3cb38 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:08.284867 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee633e55-d993-471e-8464-ef373bf64536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:08.285543 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee633e55-d993-471e-8464-ef373bf64536 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:08.285819 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee633e55-d993-471e-8464-ef373bf64536 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:08.286135 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee633e55-d993-471e-8464-ef373bf64536 admin admin] PATH_INFO: `/v3/projects/ddebfbe211414bca924d40cbf358a020/users/663e1fee12c54a6f92ffe12d1063d6fe/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:08.309930 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee633e55-d993-471e-8464-ef373bf64536 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddebfbe211414bca924d40cbf358a020, user_id=663e1fee12c54a6f92ffe12d1063d6fe, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:08.313332 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee633e55-d993-471e-8464-ef373bf64536 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:08.331302 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:08.342320 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1366/2717] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:49:08 2022] PUT /identity/v3/projects/ddebfbe211414bca924d40cbf358a020/users/663e1fee12c54a6f92ffe12d1063d6fe/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:49:08.352894 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83169566-416b-4ad5-90d8-b9052ab41a7b None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:08.365893 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83169566-416b-4ad5-90d8-b9052ab41a7b None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:08.365893 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83169566-416b-4ad5-90d8-b9052ab41a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:08.410399 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-83169566-416b-4ad5-90d8-b9052ab41a7b None None] MFA Rules not processed for user `663e1fee12c54a6f92ffe12d1063d6fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:08.413670 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-83169566-416b-4ad5-90d8-b9052ab41a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:08.524690 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:08.569647 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1352/2718] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:49:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:49:08.585263 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee633e55-d993-471e-8464-ef373bf64536 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:08.587251 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ee633e55-d993-471e-8464-ef373bf64536 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:08.762205 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee633e55-d993-471e-8464-ef373bf64536 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:08.773317 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd506426-58c4-4792-bdfe-883ea5a89e0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf14b64bf04b2ba74d2fe1fd41ff71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:08.774160 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd506426-58c4-4792-bdfe-883ea5a89e0a service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:08.774534 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd506426-58c4-4792-bdfe-883ea5a89e0a service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:08.775018 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd506426-58c4-4792-bdfe-883ea5a89e0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:08.788722 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd506426-58c4-4792-bdfe-883ea5a89e0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:08.790494 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd506426-58c4-4792-bdfe-883ea5a89e0a service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:08.807425 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1367/2719] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:49:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:49:10.644373 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc78f1-d243-4892-8572-780994396aae None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:10.644850 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc78f1-d243-4892-8572-780994396aae None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:10.644952 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc78f1-d243-4892-8572-780994396aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:10.712783 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-c7dc78f1-d243-4892-8572-780994396aae None None] MFA Rules not processed for user `a7a959e57e0c4713a855844645fdc26a`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:10.727970 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c7dc78f1-d243-4892-8572-780994396aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:10.851696 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1353/2720] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:49:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:49:10.856432 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd506426-58c4-4792-bdfe-883ea5a89e0a service nova] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:10.858011 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-dd506426-58c4-4792-bdfe-883ea5a89e0a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:10.927255 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd506426-58c4-4792-bdfe-883ea5a89e0a service nova] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:10.931815 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bcd25b-f2e3-4bff-a957-0c539cc5ea95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:10.932420 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bcd25b-f2e3-4bff-a957-0c539cc5ea95 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:10.932702 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bcd25b-f2e3-4bff-a957-0c539cc5ea95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:10.932977 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bcd25b-f2e3-4bff-a957-0c539cc5ea95 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:10.933609 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bcd25b-f2e3-4bff-a957-0c539cc5ea95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:10.935555 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bcd25b-f2e3-4bff-a957-0c539cc5ea95 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:10.963780 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1368/2721] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:49:10 2022] POST /identity/v3/projects => generated 385 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:49:10.967999 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc78f1-d243-4892-8572-780994396aae None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:10.979342 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc78f1-d243-4892-8572-780994396aae None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:11.042579 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd5a3a-e5b0-4230-8da7-67e7b9835a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:11.043195 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd5a3a-e5b0-4230-8da7-67e7b9835a23 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:11.043475 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd5a3a-e5b0-4230-8da7-67e7b9835a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:11.043743 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd5a3a-e5b0-4230-8da7-67e7b9835a23 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:11.044663 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd5a3a-e5b0-4230-8da7-67e7b9835a23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:11.048479 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd5a3a-e5b0-4230-8da7-67e7b9835a23 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:11.077875 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1354/2722] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:49:10 2022] POST /identity/v3/users => generated 299 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:49:11.079090 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bcd25b-f2e3-4bff-a957-0c539cc5ea95 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:11.092564 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bcd25b-f2e3-4bff-a957-0c539cc5ea95 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:11.097784 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dec65d-7866-49b7-a6e1-63b36466c3c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:11.098352 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dec65d-7866-49b7-a6e1-63b36466c3c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:11.098630 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dec65d-7866-49b7-a6e1-63b36466c3c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:11.098899 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dec65d-7866-49b7-a6e1-63b36466c3c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:11.099523 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dec65d-7866-49b7-a6e1-63b36466c3c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:11.101107 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dec65d-7866-49b7-a6e1-63b36466c3c9 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:11.111003 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1369/2723] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:49:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:49:11.116203 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd5a3a-e5b0-4230-8da7-67e7b9835a23 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:11.127222 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd5a3a-e5b0-4230-8da7-67e7b9835a23 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:11.132907 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6e25a6-e9b1-4f04-b5ca-0f95ceebf4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:11.133563 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e25a6-e9b1-4f04-b5ca-0f95ceebf4ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:11.133841 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e25a6-e9b1-4f04-b5ca-0f95ceebf4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:11.134144 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e25a6-e9b1-4f04-b5ca-0f95ceebf4ce admin admin] PATH_INFO: `/v3/projects/50b7a9ed7d494df39cc0040c1ca761b8/users/2147ccd32c5b45d8bcea965ff5c5a849/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:11.154097 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6e25a6-e9b1-4f04-b5ca-0f95ceebf4ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=50b7a9ed7d494df39cc0040c1ca761b8, user_id=2147ccd32c5b45d8bcea965ff5c5a849, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:11.160038 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6e25a6-e9b1-4f04-b5ca-0f95ceebf4ce admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:11.189710 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1355/2724] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:49:11 2022] PUT /identity/v3/projects/50b7a9ed7d494df39cc0040c1ca761b8/users/2147ccd32c5b45d8bcea965ff5c5a849/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:49:11.194599 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dec65d-7866-49b7-a6e1-63b36466c3c9 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:11.206620 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dec65d-7866-49b7-a6e1-63b36466c3c9 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:11.314614 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3c1473-fa8d-4bb1-b41e-bbd95278e19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:11.315225 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c1473-fa8d-4bb1-b41e-bbd95278e19c admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:11.315598 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c1473-fa8d-4bb1-b41e-bbd95278e19c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:11.315931 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c1473-fa8d-4bb1-b41e-bbd95278e19c admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:11.316636 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3c1473-fa8d-4bb1-b41e-bbd95278e19c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:11.318348 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3c1473-fa8d-4bb1-b41e-bbd95278e19c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:11.320159 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:11.335909 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1370/2725] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:49:11 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:49:11.338693 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6e25a6-e9b1-4f04-b5ca-0f95ceebf4ce admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:11.352106 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6e25a6-e9b1-4f04-b5ca-0f95ceebf4ce admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:11.356915 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a314c7c-917e-462d-a944-46dd5cd600ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:11.357629 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a314c7c-917e-462d-a944-46dd5cd600ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:11.358002 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a314c7c-917e-462d-a944-46dd5cd600ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:11.358357 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a314c7c-917e-462d-a944-46dd5cd600ee admin admin] PATH_INFO: `/v3/projects/50b7a9ed7d494df39cc0040c1ca761b8/users/2147ccd32c5b45d8bcea965ff5c5a849/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:11.360721 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a314c7c-917e-462d-a944-46dd5cd600ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=50b7a9ed7d494df39cc0040c1ca761b8, user_id=2147ccd32c5b45d8bcea965ff5c5a849, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:11.364965 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a314c7c-917e-462d-a944-46dd5cd600ee admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:11.377876 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1356/2726] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:49:11 2022] PUT /identity/v3/projects/50b7a9ed7d494df39cc0040c1ca761b8/users/2147ccd32c5b45d8bcea965ff5c5a849/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:49:11.974653 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3c1473-fa8d-4bb1-b41e-bbd95278e19c admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:11.985529 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3c1473-fa8d-4bb1-b41e-bbd95278e19c admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:12.072852 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240147a5-8fdd-417d-88c8-f55f3ab3ec50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:12.073909 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240147a5-8fdd-417d-88c8-f55f3ab3ec50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:12.074482 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240147a5-8fdd-417d-88c8-f55f3ab3ec50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:12.074934 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240147a5-8fdd-417d-88c8-f55f3ab3ec50 admin admin] PATH_INFO: `/v3/users/663e1fee12c54a6f92ffe12d1063d6fe` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:12.076315 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240147a5-8fdd-417d-88c8-f55f3ab3ec50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=663e1fee12c54a6f92ffe12d1063d6fe)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:12.078629 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240147a5-8fdd-417d-88c8-f55f3ab3ec50 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:12.163593 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-240147a5-8fdd-417d-88c8-f55f3ab3ec50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '663e1fee12c54a6f92ffe12d1063d6fe'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:12.164360 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-240147a5-8fdd-417d-88c8-f55f3ab3ec50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '663e1fee12c54a6f92ffe12d1063d6fe'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:12.180806 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-240147a5-8fdd-417d-88c8-f55f3ab3ec50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '663e1fee12c54a6f92ffe12d1063d6fe'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:12.194840 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-240147a5-8fdd-417d-88c8-f55f3ab3ec50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '663e1fee12c54a6f92ffe12d1063d6fe'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:12.204632 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-240147a5-8fdd-417d-88c8-f55f3ab3ec50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '663e1fee12c54a6f92ffe12d1063d6fe'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:12.211901 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1371/2727] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:49:11 2022] DELETE /identity/v3/users/663e1fee12c54a6f92ffe12d1063d6fe => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:49:12.216677 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6159ac1-7473-48c8-a1df-bab5a45b31ad None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:12.216677 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6159ac1-7473-48c8-a1df-bab5a45b31ad None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:12.217260 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6159ac1-7473-48c8-a1df-bab5a45b31ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:12.236930 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-f6159ac1-7473-48c8-a1df-bab5a45b31ad None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:12.239104 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-f6159ac1-7473-48c8-a1df-bab5a45b31ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:12.269657 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:12.331533 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1357/2728] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:49:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:49:12.340182 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240147a5-8fdd-417d-88c8-f55f3ab3ec50 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:12.342527 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-240147a5-8fdd-417d-88c8-f55f3ab3ec50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:12.408467 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240147a5-8fdd-417d-88c8-f55f3ab3ec50 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:12.412223 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23834f23-c6ca-49fa-9dc3-a3b6689ba0e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:12.413378 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23834f23-c6ca-49fa-9dc3-a3b6689ba0e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:12.413671 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23834f23-c6ca-49fa-9dc3-a3b6689ba0e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:12.414003 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23834f23-c6ca-49fa-9dc3-a3b6689ba0e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:12.428860 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23834f23-c6ca-49fa-9dc3-a3b6689ba0e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:12.431259 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23834f23-c6ca-49fa-9dc3-a3b6689ba0e1 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:12.461802 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1372/2729] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:49:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:49:12.809781 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6159ac1-7473-48c8-a1df-bab5a45b31ad None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:12.812692 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-f6159ac1-7473-48c8-a1df-bab5a45b31ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:12.848146 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6159ac1-7473-48c8-a1df-bab5a45b31ad None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:12.863372 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83411ec1-d60c-4746-b698-90d741f1d9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:12.864156 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83411ec1-d60c-4746-b698-90d741f1d9dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:12.864498 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83411ec1-d60c-4746-b698-90d741f1d9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:12.864856 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83411ec1-d60c-4746-b698-90d741f1d9dc admin admin] PATH_INFO: `/v3/projects/ddebfbe211414bca924d40cbf358a020` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:12.866209 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83411ec1-d60c-4746-b698-90d741f1d9dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddebfbe211414bca924d40cbf358a020)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:12.870669 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83411ec1-d60c-4746-b698-90d741f1d9dc admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:12.931352 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-83411ec1-d60c-4746-b698-90d741f1d9dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddebfbe211414bca924d40cbf358a020', 'request_id': 'req-83411ec1-d60c-4746-b698-90d741f1d9dc'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:12.942771 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-83411ec1-d60c-4746-b698-90d741f1d9dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddebfbe211414bca924d40cbf358a020', 'request_id': 'req-83411ec1-d60c-4746-b698-90d741f1d9dc'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:12.951798 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-83411ec1-d60c-4746-b698-90d741f1d9dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddebfbe211414bca924d40cbf358a020', 'request_id': 'req-83411ec1-d60c-4746-b698-90d741f1d9dc'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:12.967926 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-83411ec1-d60c-4746-b698-90d741f1d9dc admin admin] The token cache is being invalidate because project ddebfbe211414bca924d40cbf358a020 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:49:12.968292 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-83411ec1-d60c-4746-b698-90d741f1d9dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:12.969198 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-83411ec1-d60c-4746-b698-90d741f1d9dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:12.971130 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1358/2730] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:49:12 2022] DELETE /identity/v3/projects/ddebfbe211414bca924d40cbf358a020 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:49:12.976800 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641eda2b-b917-4658-96d1-5c822bebdc1b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:12.977293 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641eda2b-b917-4658-96d1-5c822bebdc1b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:12.977399 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641eda2b-b917-4658-96d1-5c822bebdc1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:13.001099 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-641eda2b-b917-4658-96d1-5c822bebdc1b None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:13.003194 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-641eda2b-b917-4658-96d1-5c822bebdc1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:13.021418 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:13.115153 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1373/2731] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:49:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:49:13.127430 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd979544-2c20-4bc1-9747-36d5f4361358 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:13.127715 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd979544-2c20-4bc1-9747-36d5f4361358 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:13.128161 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd979544-2c20-4bc1-9747-36d5f4361358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:13.184171 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-dd979544-2c20-4bc1-9747-36d5f4361358 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:13.185341 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-dd979544-2c20-4bc1-9747-36d5f4361358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:13.197598 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1359/2732] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:49:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:49:13.198728 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641eda2b-b917-4658-96d1-5c822bebdc1b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:13.209513 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641eda2b-b917-4658-96d1-5c822bebdc1b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:13.216204 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fc8ebd-d7af-4584-8432-fd8b78fb0944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:13.216832 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc8ebd-d7af-4584-8432-fd8b78fb0944 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:13.217109 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc8ebd-d7af-4584-8432-fd8b78fb0944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:13.217385 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc8ebd-d7af-4584-8432-fd8b78fb0944 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:13.218058 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fc8ebd-d7af-4584-8432-fd8b78fb0944 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:49:13.218772 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fc8ebd-d7af-4584-8432-fd8b78fb0944 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:13.220799 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fc8ebd-d7af-4584-8432-fd8b78fb0944 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:13.238913 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1374/2733] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:49:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:49:13.243269 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd979544-2c20-4bc1-9747-36d5f4361358 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:13.253034 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd979544-2c20-4bc1-9747-36d5f4361358 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:13.258944 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58536b4a-81de-439d-b8c4-584e2903c693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:13.259580 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58536b4a-81de-439d-b8c4-584e2903c693 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:13.259849 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58536b4a-81de-439d-b8c4-584e2903c693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:13.260191 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58536b4a-81de-439d-b8c4-584e2903c693 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:13.261186 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58536b4a-81de-439d-b8c4-584e2903c693 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:13.263219 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58536b4a-81de-439d-b8c4-584e2903c693 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:13.303633 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1360/2734] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:49:13 2022] POST /identity/v3/projects => generated 369 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:49:13.310905 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fc8ebd-d7af-4584-8432-fd8b78fb0944 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:13.336220 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fc8ebd-d7af-4584-8432-fd8b78fb0944 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:13.435810 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b133a97b-9b5e-4e2b-94c5-07e80405d565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:13.436322 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133a97b-9b5e-4e2b-94c5-07e80405d565 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:13.436600 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133a97b-9b5e-4e2b-94c5-07e80405d565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:13.436871 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133a97b-9b5e-4e2b-94c5-07e80405d565 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:13.437729 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b133a97b-9b5e-4e2b-94c5-07e80405d565 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:13.441810 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b133a97b-9b5e-4e2b-94c5-07e80405d565 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:13.473217 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1375/2735] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:49:13 2022] POST /identity/v3/users => generated 285 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:49:13.474403 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58536b4a-81de-439d-b8c4-584e2903c693 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:13.489226 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58536b4a-81de-439d-b8c4-584e2903c693 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:13.508464 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47db37d-ece6-49e5-9ae1-9682b43be6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:13.509290 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47db37d-ece6-49e5-9ae1-9682b43be6e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:13.509713 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47db37d-ece6-49e5-9ae1-9682b43be6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:13.510176 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47db37d-ece6-49e5-9ae1-9682b43be6e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:13.510977 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47db37d-ece6-49e5-9ae1-9682b43be6e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:13.514563 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47db37d-ece6-49e5-9ae1-9682b43be6e4 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:13.532897 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1361/2736] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:49:13 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:49:13.537127 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b133a97b-9b5e-4e2b-94c5-07e80405d565 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:13.556743 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b133a97b-9b5e-4e2b-94c5-07e80405d565 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:13.576704 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64702edf-a640-4afa-bea0-a576e8779a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:13.577333 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64702edf-a640-4afa-bea0-a576e8779a67 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:13.577614 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64702edf-a640-4afa-bea0-a576e8779a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:13.577915 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64702edf-a640-4afa-bea0-a576e8779a67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:13.578558 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64702edf-a640-4afa-bea0-a576e8779a67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:13.580160 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64702edf-a640-4afa-bea0-a576e8779a67 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:13.591888 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1376/2737] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:49:13 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:49:13.595757 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47db37d-ece6-49e5-9ae1-9682b43be6e4 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:13.606975 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47db37d-ece6-49e5-9ae1-9682b43be6e4 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:13.613547 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33deed2-4aa7-4164-98b1-45cd29cc0de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:13.614674 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33deed2-4aa7-4164-98b1-45cd29cc0de7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:13.615020 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33deed2-4aa7-4164-98b1-45cd29cc0de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:13.615286 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33deed2-4aa7-4164-98b1-45cd29cc0de7 admin admin] PATH_INFO: `/v3/projects/d9d44532365c4f3fb4ee663da0daafff/users/0bb24f956dce4aaba5b3f29fa071a251/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:13.638002 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33deed2-4aa7-4164-98b1-45cd29cc0de7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9d44532365c4f3fb4ee663da0daafff, user_id=0bb24f956dce4aaba5b3f29fa071a251, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:13.640827 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33deed2-4aa7-4164-98b1-45cd29cc0de7 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:13.663685 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1362/2738] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:49:13 2022] PUT /identity/v3/projects/d9d44532365c4f3fb4ee663da0daafff/users/0bb24f956dce4aaba5b3f29fa071a251/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:49:13.674065 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff131ab2-aafa-4fa8-8c0b-f93471cb87b6 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:13.674346 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff131ab2-aafa-4fa8-8c0b-f93471cb87b6 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:13.674618 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff131ab2-aafa-4fa8-8c0b-f93471cb87b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:13.717176 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-ff131ab2-aafa-4fa8-8c0b-f93471cb87b6 None None] MFA Rules not processed for user `0bb24f956dce4aaba5b3f29fa071a251`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:13.718425 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ff131ab2-aafa-4fa8-8c0b-f93471cb87b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:13.821299 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1377/2739] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:49:13 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:49:13.825500 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33deed2-4aa7-4164-98b1-45cd29cc0de7 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:13.890142 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff131ab2-aafa-4fa8-8c0b-f93471cb87b6 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:13.893676 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ff131ab2-aafa-4fa8-8c0b-f93471cb87b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:13.917270 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:13.962535 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:13.995129 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33deed2-4aa7-4164-98b1-45cd29cc0de7 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:14.004155 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefd4f40-344e-4e0c-ae24-6984c4179820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:14.004863 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefd4f40-344e-4e0c-ae24-6984c4179820 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:14.005208 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefd4f40-344e-4e0c-ae24-6984c4179820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:14.005537 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefd4f40-344e-4e0c-ae24-6984c4179820 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:14.006301 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefd4f40-344e-4e0c-ae24-6984c4179820 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:14.008304 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefd4f40-344e-4e0c-ae24-6984c4179820 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:14.029012 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff131ab2-aafa-4fa8-8c0b-f93471cb87b6 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:14.033349 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f7c48-80af-4845-86c6-feb6d37beb16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:14.034703 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f7c48-80af-4845-86c6-feb6d37beb16 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:14.035220 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f7c48-80af-4845-86c6-feb6d37beb16 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:14.035481 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f7c48-80af-4845-86c6-feb6d37beb16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:14.037618 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7e7f7c48-80af-4845-86c6-feb6d37beb16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:14.052866 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1363/2740] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:49:13 2022] POST /identity/v3/projects => generated 367 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:49:14.054229 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefd4f40-344e-4e0c-ae24-6984c4179820 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:14.146878 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7f7c48-80af-4845-86c6-feb6d37beb16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:14.150427 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7f7c48-80af-4845-86c6-feb6d37beb16 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:14.174074 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefd4f40-344e-4e0c-ae24-6984c4179820 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:14.182932 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44aabae-fba8-4ff4-9a3b-071faa9f196b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:14.183948 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44aabae-fba8-4ff4-9a3b-071faa9f196b admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:14.185197 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44aabae-fba8-4ff4-9a3b-071faa9f196b admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:14.185708 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44aabae-fba8-4ff4-9a3b-071faa9f196b admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:14.186818 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44aabae-fba8-4ff4-9a3b-071faa9f196b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:14.190129 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1378/2741] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:49:13 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:49:14.222407 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44aabae-fba8-4ff4-9a3b-071faa9f196b admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:14.270213 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f7c48-80af-4845-86c6-feb6d37beb16 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:14.272772 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7e7f7c48-80af-4845-86c6-feb6d37beb16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:14.274260 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1364/2742] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:49:14 2022] POST /identity/v3/users => generated 290 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:49:14.279317 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44aabae-fba8-4ff4-9a3b-071faa9f196b admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:14.302685 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44aabae-fba8-4ff4-9a3b-071faa9f196b admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:14.311699 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f7c48-80af-4845-86c6-feb6d37beb16 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:14.319394 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1371ea-89ec-431d-8ea0-57d2bf1a7cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:14.319981 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1371ea-89ec-431d-8ea0-57d2bf1a7cb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:14.320268 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1371ea-89ec-431d-8ea0-57d2bf1a7cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:14.323595 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe088122-a941-414d-b0f6-4364010697d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:14.324354 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe088122-a941-414d-b0f6-4364010697d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:14.325759 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1371ea-89ec-431d-8ea0-57d2bf1a7cb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:14.326807 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1371ea-89ec-431d-8ea0-57d2bf1a7cb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:14.328001 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe088122-a941-414d-b0f6-4364010697d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:14.328301 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe088122-a941-414d-b0f6-4364010697d3 admin admin] PATH_INFO: `/v3/users/83fef35a221e4ca089336d624e794247` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:14.328724 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1371ea-89ec-431d-8ea0-57d2bf1a7cb4 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:14.339312 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe088122-a941-414d-b0f6-4364010697d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83fef35a221e4ca089336d624e794247)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:14.346336 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe088122-a941-414d-b0f6-4364010697d3 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:14.361536 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1365/2743] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:49:14.363304 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1371ea-89ec-431d-8ea0-57d2bf1a7cb4 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:14.422400 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1371ea-89ec-431d-8ea0-57d2bf1a7cb4 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:14.438174 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464e0f7-f4a1-4629-8d63-f4a7a2ab7ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:14.439270 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464e0f7-f4a1-4629-8d63-f4a7a2ab7ea3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:14.439805 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464e0f7-f4a1-4629-8d63-f4a7a2ab7ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:14.440327 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464e0f7-f4a1-4629-8d63-f4a7a2ab7ea3 admin admin] PATH_INFO: `/v3/projects/982a3e97ef29446f98c3fe74c68a9099/users/f5664ef86e22410aba1f3b62e999fff0/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:14.496673 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7464e0f7-f4a1-4629-8d63-f4a7a2ab7ea3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=982a3e97ef29446f98c3fe74c68a9099, user_id=f5664ef86e22410aba1f3b62e999fff0, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:14.499648 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7464e0f7-f4a1-4629-8d63-f4a7a2ab7ea3 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:14.505883 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-fe088122-a941-414d-b0f6-4364010697d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83fef35a221e4ca089336d624e794247'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:14.508653 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-fe088122-a941-414d-b0f6-4364010697d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83fef35a221e4ca089336d624e794247'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:14.539967 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1366/2744] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:49:14 2022] PUT /identity/v3/projects/982a3e97ef29446f98c3fe74c68a9099/users/f5664ef86e22410aba1f3b62e999fff0/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:49:14.544309 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464e0f7-f4a1-4629-8d63-f4a7a2ab7ea3 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:14.547336 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-fe088122-a941-414d-b0f6-4364010697d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83fef35a221e4ca089336d624e794247'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:14.568437 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-fe088122-a941-414d-b0f6-4364010697d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83fef35a221e4ca089336d624e794247'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:14.578170 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-fe088122-a941-414d-b0f6-4364010697d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83fef35a221e4ca089336d624e794247'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:14.592174 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1379/2745] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:49:14 2022] DELETE /identity/v3/users/83fef35a221e4ca089336d624e794247 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:49:14.597675 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee68a2-d1f5-48ba-a691-7be34f4c42ff None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:14.599365 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee68a2-d1f5-48ba-a691-7be34f4c42ff None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:14.599873 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee68a2-d1f5-48ba-a691-7be34f4c42ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:14.667828 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-b4ee68a2-d1f5-48ba-a691-7be34f4c42ff None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:14.670041 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b4ee68a2-d1f5-48ba-a691-7be34f4c42ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:14.715232 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464e0f7-f4a1-4629-8d63-f4a7a2ab7ea3 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:14.740634 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:14.789897 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1380/2746] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:49:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:49:14.806865 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ee68a2-d1f5-48ba-a691-7be34f4c42ff None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:14.808095 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:14.810353 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b4ee68a2-d1f5-48ba-a691-7be34f4c42ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:14.872857 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beba8a53-d665-4ca9-9063-b6bc2acf1bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:14.873479 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beba8a53-d665-4ca9-9063-b6bc2acf1bd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:14.873753 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beba8a53-d665-4ca9-9063-b6bc2acf1bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:14.874060 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beba8a53-d665-4ca9-9063-b6bc2acf1bd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:14.875006 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beba8a53-d665-4ca9-9063-b6bc2acf1bd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:14.876633 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beba8a53-d665-4ca9-9063-b6bc2acf1bd6 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:14.903563 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1367/2747] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:49:14.906461 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beba8a53-d665-4ca9-9063-b6bc2acf1bd6 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:14.910657 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-beba8a53-d665-4ca9-9063-b6bc2acf1bd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:14.936918 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ee68a2-d1f5-48ba-a691-7be34f4c42ff None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:14.939035 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beba8a53-d665-4ca9-9063-b6bc2acf1bd6 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:14.943212 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61ff091-34cc-4d71-9415-69f4c1fe2c05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:14.945633 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ff091-34cc-4d71-9415-69f4c1fe2c05 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:14.946152 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ff091-34cc-4d71-9415-69f4c1fe2c05 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:14.949514 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ff091-34cc-4d71-9415-69f4c1fe2c05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:14.953725 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5621422-cedb-4202-bf0e-3c861631d5be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:14.954485 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5621422-cedb-4202-bf0e-3c861631d5be admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:14.954760 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5621422-cedb-4202-bf0e-3c861631d5be admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:14.955065 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5621422-cedb-4202-bf0e-3c861631d5be admin admin] PATH_INFO: `/v3/projects/982a3e97ef29446f98c3fe74c68a9099/users/f5664ef86e22410aba1f3b62e999fff0/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:14.959580 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5621422-cedb-4202-bf0e-3c861631d5be admin admin] RBAC: Authorizing `identity:create_grant(project_id=982a3e97ef29446f98c3fe74c68a9099, user_id=f5664ef86e22410aba1f3b62e999fff0, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:14.970703 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5621422-cedb-4202-bf0e-3c861631d5be admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:14.974475 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61ff091-34cc-4d71-9415-69f4c1fe2c05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:14.976057 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61ff091-34cc-4d71-9415-69f4c1fe2c05 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:14.994727 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1368/2748] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:49:14 2022] PUT /identity/v3/projects/982a3e97ef29446f98c3fe74c68a9099/users/f5664ef86e22410aba1f3b62e999fff0/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:49:15.005702 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686f452-5a87-4a82-ac26-ea6a28bba5d8 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:15.006294 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686f452-5a87-4a82-ac26-ea6a28bba5d8 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:15.006588 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686f452-5a87-4a82-ac26-ea6a28bba5d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:15.009986 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1381/2749] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:49:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:49:15.072391 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-e686f452-5a87-4a82-ac26-ea6a28bba5d8 None None] MFA Rules not processed for user `f5664ef86e22410aba1f3b62e999fff0`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:15.074131 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e686f452-5a87-4a82-ac26-ea6a28bba5d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:15.218504 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1369/2750] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:49:15 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:49:15.231216 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61ff091-34cc-4d71-9415-69f4c1fe2c05 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:15.233321 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f61ff091-34cc-4d71-9415-69f4c1fe2c05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:15.450189 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61ff091-34cc-4d71-9415-69f4c1fe2c05 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:15.590439 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb1e66f-60f2-4ac9-8584-92de51c9963c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:15.591070 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1e66f-60f2-4ac9-8584-92de51c9963c service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:15.591344 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1e66f-60f2-4ac9-8584-92de51c9963c service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:15.591606 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1e66f-60f2-4ac9-8584-92de51c9963c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:15.609719 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb1e66f-60f2-4ac9-8584-92de51c9963c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:15.611224 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb1e66f-60f2-4ac9-8584-92de51c9963c service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:15.644425 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1382/2751] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:49:15 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:49:16.363958 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e686f452-5a87-4a82-ac26-ea6a28bba5d8 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:16.370010 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e686f452-5a87-4a82-ac26-ea6a28bba5d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:16.542425 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e686f452-5a87-4a82-ac26-ea6a28bba5d8 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:16.553118 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf9a656-c1ae-4910-8157-4c420f1425b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:16.555426 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9a656-c1ae-4910-8157-4c420f1425b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:16.555426 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9a656-c1ae-4910-8157-4c420f1425b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:16.555426 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9a656-c1ae-4910-8157-4c420f1425b3 admin admin] PATH_INFO: `/v3/projects/8ff3f4275a4241119e7d42d93cedb907` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:16.557272 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf9a656-c1ae-4910-8157-4c420f1425b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ff3f4275a4241119e7d42d93cedb907)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:16.559174 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf9a656-c1ae-4910-8157-4c420f1425b3 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:16.718200 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:16.727322 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ebf9a656-c1ae-4910-8157-4c420f1425b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ff3f4275a4241119e7d42d93cedb907', 'request_id': 'req-ebf9a656-c1ae-4910-8157-4c420f1425b3'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:16.745598 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ebf9a656-c1ae-4910-8157-4c420f1425b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ff3f4275a4241119e7d42d93cedb907', 'request_id': 'req-ebf9a656-c1ae-4910-8157-4c420f1425b3'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:16.757866 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ebf9a656-c1ae-4910-8157-4c420f1425b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ff3f4275a4241119e7d42d93cedb907', 'request_id': 'req-ebf9a656-c1ae-4910-8157-4c420f1425b3'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:16.787502 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ebf9a656-c1ae-4910-8157-4c420f1425b3 admin admin] The token cache is being invalidate because project 8ff3f4275a4241119e7d42d93cedb907 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:49:16.788006 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ebf9a656-c1ae-4910-8157-4c420f1425b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:16.790740 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ebf9a656-c1ae-4910-8157-4c420f1425b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:16.795878 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1370/2752] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:49:16 2022] DELETE /identity/v3/projects/8ff3f4275a4241119e7d42d93cedb907 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:49:16.801440 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6238da-bdf8-44df-b595-7ea8e781b621 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:16.801959 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6238da-bdf8-44df-b595-7ea8e781b621 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:16.802174 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6238da-bdf8-44df-b595-7ea8e781b621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:16.850535 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-5d6238da-bdf8-44df-b595-7ea8e781b621 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:16.852007 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-5d6238da-bdf8-44df-b595-7ea8e781b621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:16.919784 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:17.001883 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1383/2753] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:49:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:49:17.067498 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3284357-4130-4703-968b-5690cdddd14f None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:17.067779 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3284357-4130-4703-968b-5690cdddd14f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:17.068055 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3284357-4130-4703-968b-5690cdddd14f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:17.101288 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-e3284357-4130-4703-968b-5690cdddd14f None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:17.102677 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e3284357-4130-4703-968b-5690cdddd14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:17.118657 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1371/2754] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:49:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:49:17.123041 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6238da-bdf8-44df-b595-7ea8e781b621 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:17.134743 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6238da-bdf8-44df-b595-7ea8e781b621 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:17.140556 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c3f325-cdb8-49a6-840f-3a1bcbc06806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:17.141293 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c3f325-cdb8-49a6-840f-3a1bcbc06806 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:17.141805 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c3f325-cdb8-49a6-840f-3a1bcbc06806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:17.142187 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c3f325-cdb8-49a6-840f-3a1bcbc06806 admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:17.157138 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c3f325-cdb8-49a6-840f-3a1bcbc06806 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:49:17.157457 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c3f325-cdb8-49a6-840f-3a1bcbc06806 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:17.161159 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c3f325-cdb8-49a6-840f-3a1bcbc06806 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:17.188606 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3284357-4130-4703-968b-5690cdddd14f None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:17.195511 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1384/2755] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:49:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:49:17.199848 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3284357-4130-4703-968b-5690cdddd14f None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:17.204980 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5eccde-b058-4c48-9ec6-f17f5bdd12c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:17.205688 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5eccde-b058-4c48-9ec6-f17f5bdd12c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:17.206836 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5eccde-b058-4c48-9ec6-f17f5bdd12c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:17.207177 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5eccde-b058-4c48-9ec6-f17f5bdd12c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:17.208035 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5eccde-b058-4c48-9ec6-f17f5bdd12c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:17.210050 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5eccde-b058-4c48-9ec6-f17f5bdd12c7 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:17.280553 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1372/2756] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:49:17 2022] POST /identity/v3/projects => generated 377 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:49:17.285197 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c3f325-cdb8-49a6-840f-3a1bcbc06806 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:17.326515 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c3f325-cdb8-49a6-840f-3a1bcbc06806 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:17.514499 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd78648-2896-4c20-bb7b-72cd94c00fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:17.515138 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd78648-2896-4c20-bb7b-72cd94c00fd4 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:17.522128 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd78648-2896-4c20-bb7b-72cd94c00fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:17.523073 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd78648-2896-4c20-bb7b-72cd94c00fd4 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:17.524519 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd78648-2896-4c20-bb7b-72cd94c00fd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:17.527214 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd78648-2896-4c20-bb7b-72cd94c00fd4 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:17.567014 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5eccde-b058-4c48-9ec6-f17f5bdd12c7 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:17.573935 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1385/2757] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:49:17 2022] POST /identity/v3/users => generated 289 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:49:17.598324 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5eccde-b058-4c48-9ec6-f17f5bdd12c7 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:17.612454 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d3ea7c-7c3e-4a73-974f-69a76511df9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:17.613095 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d3ea7c-7c3e-4a73-974f-69a76511df9f admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:17.613374 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d3ea7c-7c3e-4a73-974f-69a76511df9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:17.613640 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d3ea7c-7c3e-4a73-974f-69a76511df9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:17.614317 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d3ea7c-7c3e-4a73-974f-69a76511df9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:17.625715 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d3ea7c-7c3e-4a73-974f-69a76511df9f admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:17.657025 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1373/2758] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:49:17 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:49:17.662079 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd78648-2896-4c20-bb7b-72cd94c00fd4 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:17.675656 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd78648-2896-4c20-bb7b-72cd94c00fd4 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:17.687298 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64026057-9417-46db-bf3d-f224a99b1cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:17.687937 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64026057-9417-46db-bf3d-f224a99b1cc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:17.688222 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64026057-9417-46db-bf3d-f224a99b1cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:17.688495 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64026057-9417-46db-bf3d-f224a99b1cc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:17.689138 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64026057-9417-46db-bf3d-f224a99b1cc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:17.690864 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64026057-9417-46db-bf3d-f224a99b1cc2 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:17.733608 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d3ea7c-7c3e-4a73-974f-69a76511df9f admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:17.743900 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1386/2759] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:49:17 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:49:17.746459 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d3ea7c-7c3e-4a73-974f-69a76511df9f admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:17.752259 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa90de9-ccc5-4363-9e0a-7503852fe3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:17.753871 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa90de9-ccc5-4363-9e0a-7503852fe3b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:17.754144 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa90de9-ccc5-4363-9e0a-7503852fe3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:17.754418 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa90de9-ccc5-4363-9e0a-7503852fe3b1 admin admin] PATH_INFO: `/v3/projects/35cf5a9c74e44a918e45867c6570aa11/users/5fac9ab7665b4f14936adea96ae22ec4/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:17.781142 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa90de9-ccc5-4363-9e0a-7503852fe3b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35cf5a9c74e44a918e45867c6570aa11, user_id=5fac9ab7665b4f14936adea96ae22ec4, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:17.785595 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa90de9-ccc5-4363-9e0a-7503852fe3b1 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:17.831657 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1374/2760] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:49:17 2022] PUT /identity/v3/projects/35cf5a9c74e44a918e45867c6570aa11/users/5fac9ab7665b4f14936adea96ae22ec4/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:49:17.837287 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c289b18-a310-488b-a4e8-9a3d4deea3ad None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:17.837958 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c289b18-a310-488b-a4e8-9a3d4deea3ad None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:17.837958 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c289b18-a310-488b-a4e8-9a3d4deea3ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:17.880627 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-8c289b18-a310-488b-a4e8-9a3d4deea3ad None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:17.881912 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8c289b18-a310-488b-a4e8-9a3d4deea3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:18.017905 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1387/2761] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:49:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:49:18.028127 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa90de9-ccc5-4363-9e0a-7503852fe3b1 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:18.029872 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7aa90de9-ccc5-4363-9e0a-7503852fe3b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:18.127433 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17065318-e7ed-4f71-a516-fbbc1a4b3d2b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:18.127433 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17065318-e7ed-4f71-a516-fbbc1a4b3d2b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:18.127433 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17065318-e7ed-4f71-a516-fbbc1a4b3d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:18.163121 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa90de9-ccc5-4363-9e0a-7503852fe3b1 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:18.167824 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-17065318-e7ed-4f71-a516-fbbc1a4b3d2b None None] MFA Rules not processed for user `2147ccd32c5b45d8bcea965ff5c5a849`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:18.169188 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-17065318-e7ed-4f71-a516-fbbc1a4b3d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:18.175786 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280dbd95-d2d3-459a-8c06-4c4f8a00c20f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:18.175786 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280dbd95-d2d3-459a-8c06-4c4f8a00c20f service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:18.175786 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280dbd95-d2d3-459a-8c06-4c4f8a00c20f service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:18.175786 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280dbd95-d2d3-459a-8c06-4c4f8a00c20f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:18.189327 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280dbd95-d2d3-459a-8c06-4c4f8a00c20f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:18.191157 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280dbd95-d2d3-459a-8c06-4c4f8a00c20f service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:18.209230 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1375/2762] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:49:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:49:18.268003 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:18.373946 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1388/2763] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:49:18 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:49:18.388041 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280dbd95-d2d3-459a-8c06-4c4f8a00c20f service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:18.390217 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-280dbd95-d2d3-459a-8c06-4c4f8a00c20f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:18.449071 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17065318-e7ed-4f71-a516-fbbc1a4b3d2b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:18.451346 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-17065318-e7ed-4f71-a516-fbbc1a4b3d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:18.468631 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17065318-e7ed-4f71-a516-fbbc1a4b3d2b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:18.476623 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd6958-1118-4b3e-8111-ea156389fa6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:18.477426 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd6958-1118-4b3e-8111-ea156389fa6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:18.477807 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd6958-1118-4b3e-8111-ea156389fa6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:18.478209 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd6958-1118-4b3e-8111-ea156389fa6d admin admin] PATH_INFO: `/v3/users/0bb24f956dce4aaba5b3f29fa071a251` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:18.479532 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fd6958-1118-4b3e-8111-ea156389fa6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bb24f956dce4aaba5b3f29fa071a251)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:18.481580 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fd6958-1118-4b3e-8111-ea156389fa6d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:18.537769 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280dbd95-d2d3-459a-8c06-4c4f8a00c20f service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:18.543167 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a579f7-08dd-476d-b532-375d2271e135 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6a46bbd249c4434ad7d9442754f098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:18.544020 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a579f7-08dd-476d-b532-375d2271e135 service glance] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:18.544528 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a579f7-08dd-476d-b532-375d2271e135 service glance] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:18.545037 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a579f7-08dd-476d-b532-375d2271e135 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:18.547416 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-47a579f7-08dd-476d-b532-375d2271e135 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:18.554239 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-50fd6958-1118-4b3e-8111-ea156389fa6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bb24f956dce4aaba5b3f29fa071a251'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:18.556581 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-50fd6958-1118-4b3e-8111-ea156389fa6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bb24f956dce4aaba5b3f29fa071a251'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:18.579510 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-50fd6958-1118-4b3e-8111-ea156389fa6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bb24f956dce4aaba5b3f29fa071a251'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:18.587644 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:18.594569 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-50fd6958-1118-4b3e-8111-ea156389fa6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bb24f956dce4aaba5b3f29fa071a251'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:18.620125 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-50fd6958-1118-4b3e-8111-ea156389fa6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bb24f956dce4aaba5b3f29fa071a251'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:18.636359 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1389/2764] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:49:18 2022] DELETE /identity/v3/users/0bb24f956dce4aaba5b3f29fa071a251 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:49:18.638778 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ba6c0-2172-47c1-abed-d11fa5c945e1 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:18.639068 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ba6c0-2172-47c1-abed-d11fa5c945e1 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:18.641033 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ba6c0-2172-47c1-abed-d11fa5c945e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:18.656477 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a579f7-08dd-476d-b532-375d2271e135 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:18.658101 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a579f7-08dd-476d-b532-375d2271e135 service glance] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:18.667780 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1376/2765] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:49:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:49:18.673057 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-f67ba6c0-2172-47c1-abed-d11fa5c945e1 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:18.674485 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f67ba6c0-2172-47c1-abed-d11fa5c945e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:18.747503 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1390/2766] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:49:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:49:18.766703 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a579f7-08dd-476d-b532-375d2271e135 service glance] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:18.769121 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-47a579f7-08dd-476d-b532-375d2271e135 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:18.785782 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a579f7-08dd-476d-b532-375d2271e135 service glance] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:18.856352 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6c28e7-8a24-48ed-8b44-98971e16a8e3 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:18.856914 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6c28e7-8a24-48ed-8b44-98971e16a8e3 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:18.856914 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6c28e7-8a24-48ed-8b44-98971e16a8e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:18.865327 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a74f902-dab8-4e2d-9444-c0ae9d16d266 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:18.866187 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74f902-dab8-4e2d-9444-c0ae9d16d266 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:18.866838 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74f902-dab8-4e2d-9444-c0ae9d16d266 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:18.867156 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74f902-dab8-4e2d-9444-c0ae9d16d266 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:18.882386 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-dd6c28e7-8a24-48ed-8b44-98971e16a8e3 None None] MFA Rules not processed for user `c81e5fb007614cf2992f5e02e85d6307`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:18.882947 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a74f902-dab8-4e2d-9444-c0ae9d16d266 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:18.884166 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-dd6c28e7-8a24-48ed-8b44-98971e16a8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:18.884621 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a74f902-dab8-4e2d-9444-c0ae9d16d266 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:18.897338 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1377/2767] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:49:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:49:18.970085 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:19.018272 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1391/2768] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Sep 28 15:49:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:49:19.101786 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a74f902-dab8-4e2d-9444-c0ae9d16d266 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:19.103581 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7a74f902-dab8-4e2d-9444-c0ae9d16d266 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:19.197239 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a74f902-dab8-4e2d-9444-c0ae9d16d266 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:19.204768 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d184c3-eb94-47f0-aad1-eacbd06d440f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '328051fe7607464f8f5dc0dcc01a817a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:19.205713 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d184c3-eb94-47f0-aad1-eacbd06d440f service swift] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:19.206647 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d184c3-eb94-47f0-aad1-eacbd06d440f service swift] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:19.207014 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d184c3-eb94-47f0-aad1-eacbd06d440f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:19.218542 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d184c3-eb94-47f0-aad1-eacbd06d440f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:19.222208 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d184c3-eb94-47f0-aad1-eacbd06d440f service swift] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:19.251066 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1378/2769] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 15:49:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 15:49:19.536297 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6c28e7-8a24-48ed-8b44-98971e16a8e3 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:19.540103 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-dd6c28e7-8a24-48ed-8b44-98971e16a8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:19.581797 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6c28e7-8a24-48ed-8b44-98971e16a8e3 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:19.591729 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c9fabb-4391-4497-87d5-ae8e8d242f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:19.600542 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c9fabb-4391-4497-87d5-ae8e8d242f67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:19.600864 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c9fabb-4391-4497-87d5-ae8e8d242f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:19.601166 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c9fabb-4391-4497-87d5-ae8e8d242f67 admin admin] PATH_INFO: `/v3/projects/d9d44532365c4f3fb4ee663da0daafff` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:19.615710 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c9fabb-4391-4497-87d5-ae8e8d242f67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9d44532365c4f3fb4ee663da0daafff)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:19.618432 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c9fabb-4391-4497-87d5-ae8e8d242f67 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:19.745145 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-75c9fabb-4391-4497-87d5-ae8e8d242f67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9d44532365c4f3fb4ee663da0daafff', 'request_id': 'req-75c9fabb-4391-4497-87d5-ae8e8d242f67'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:19.759882 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-75c9fabb-4391-4497-87d5-ae8e8d242f67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9d44532365c4f3fb4ee663da0daafff', 'request_id': 'req-75c9fabb-4391-4497-87d5-ae8e8d242f67'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:19.776698 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-75c9fabb-4391-4497-87d5-ae8e8d242f67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9d44532365c4f3fb4ee663da0daafff', 'request_id': 'req-75c9fabb-4391-4497-87d5-ae8e8d242f67'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:19.808518 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-75c9fabb-4391-4497-87d5-ae8e8d242f67 admin admin] The token cache is being invalidate because project d9d44532365c4f3fb4ee663da0daafff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:49:19.809094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-75c9fabb-4391-4497-87d5-ae8e8d242f67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:19.811589 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-75c9fabb-4391-4497-87d5-ae8e8d242f67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:19.814556 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1392/2770] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:49:19 2022] DELETE /identity/v3/projects/d9d44532365c4f3fb4ee663da0daafff => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:49:19.819654 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d184c3-eb94-47f0-aad1-eacbd06d440f service swift] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:19.843317 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d184c3-eb94-47f0-aad1-eacbd06d440f service swift] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:19.978598 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1b172b-bca7-4690-9964-1ddab752c637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:19.979258 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b172b-bca7-4690-9964-1ddab752c637 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:19.979538 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b172b-bca7-4690-9964-1ddab752c637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:19.979812 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b172b-bca7-4690-9964-1ddab752c637 admin admin] PATH_INFO: `/v3/users/f5664ef86e22410aba1f3b62e999fff0` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:19.981954 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1b172b-bca7-4690-9964-1ddab752c637 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5664ef86e22410aba1f3b62e999fff0)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:19.984300 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1b172b-bca7-4690-9964-1ddab752c637 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:19.988432 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=100 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:20.123446 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-cc1b172b-bca7-4690-9964-1ddab752c637 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5664ef86e22410aba1f3b62e999fff0'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:20.138380 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-cc1b172b-bca7-4690-9964-1ddab752c637 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5664ef86e22410aba1f3b62e999fff0'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:20.140646 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-cc1b172b-bca7-4690-9964-1ddab752c637 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5664ef86e22410aba1f3b62e999fff0'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:20.165574 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-cc1b172b-bca7-4690-9964-1ddab752c637 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5664ef86e22410aba1f3b62e999fff0'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:20.166316 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-cc1b172b-bca7-4690-9964-1ddab752c637 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5664ef86e22410aba1f3b62e999fff0'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:20.206261 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1379/2771] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:49:19 2022] DELETE /identity/v3/users/f5664ef86e22410aba1f3b62e999fff0 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:49:20.666044 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c9fabb-4391-4497-87d5-ae8e8d242f67 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:20.672181 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-75c9fabb-4391-4497-87d5-ae8e8d242f67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:20.700627 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c9fabb-4391-4497-87d5-ae8e8d242f67 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:20.843446 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d427815-1639-40ad-8131-af127049cfde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:20.844615 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d427815-1639-40ad-8131-af127049cfde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:20.845053 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d427815-1639-40ad-8131-af127049cfde admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:20.845516 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d427815-1639-40ad-8131-af127049cfde admin admin] PATH_INFO: `/v3/projects/982a3e97ef29446f98c3fe74c68a9099` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:20.847750 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d427815-1639-40ad-8131-af127049cfde admin admin] RBAC: Authorizing `identity:delete_project(project_id=982a3e97ef29446f98c3fe74c68a9099)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:20.850162 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d427815-1639-40ad-8131-af127049cfde admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:20.924041 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7d427815-1639-40ad-8131-af127049cfde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '982a3e97ef29446f98c3fe74c68a9099', 'request_id': 'req-7d427815-1639-40ad-8131-af127049cfde'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:20.936319 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7d427815-1639-40ad-8131-af127049cfde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '982a3e97ef29446f98c3fe74c68a9099', 'request_id': 'req-7d427815-1639-40ad-8131-af127049cfde'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:20.957669 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7d427815-1639-40ad-8131-af127049cfde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '982a3e97ef29446f98c3fe74c68a9099', 'request_id': 'req-7d427815-1639-40ad-8131-af127049cfde'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:20.975685 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:20.981419 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7d427815-1639-40ad-8131-af127049cfde admin admin] The token cache is being invalidate because project 982a3e97ef29446f98c3fe74c68a9099 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:49:20.981791 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7d427815-1639-40ad-8131-af127049cfde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:20.983117 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-7d427815-1639-40ad-8131-af127049cfde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:20.986124 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1393/2772] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:49:20 2022] DELETE /identity/v3/projects/982a3e97ef29446f98c3fe74c68a9099 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:49:20.991688 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8352d5-04b0-4519-a760-49b4ff2f5f1f None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:20.994004 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8352d5-04b0-4519-a760-49b4ff2f5f1f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:20.994004 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8352d5-04b0-4519-a760-49b4ff2f5f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:21.026349 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-df8352d5-04b0-4519-a760-49b4ff2f5f1f None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:21.027408 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-df8352d5-04b0-4519-a760-49b4ff2f5f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:21.200640 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1380/2773] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:49:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:49:21.208669 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6967d65-4b74-4e81-9298-ab2167753edf None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:21.208955 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6967d65-4b74-4e81-9298-ab2167753edf None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:21.209233 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6967d65-4b74-4e81-9298-ab2167753edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:21.253208 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-f6967d65-4b74-4e81-9298-ab2167753edf None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:21.259046 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-f6967d65-4b74-4e81-9298-ab2167753edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:21.282780 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1394/2774] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:49:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:49:21.286868 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8352d5-04b0-4519-a760-49b4ff2f5f1f None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:21.306546 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8352d5-04b0-4519-a760-49b4ff2f5f1f None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:21.313188 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f083c237-3f46-4280-bbbe-d6bcc86b6d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:21.313809 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f083c237-3f46-4280-bbbe-d6bcc86b6d18 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:21.314136 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f083c237-3f46-4280-bbbe-d6bcc86b6d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:21.314407 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f083c237-3f46-4280-bbbe-d6bcc86b6d18 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:21.316129 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f083c237-3f46-4280-bbbe-d6bcc86b6d18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:49:21.316394 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f083c237-3f46-4280-bbbe-d6bcc86b6d18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:21.317996 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f083c237-3f46-4280-bbbe-d6bcc86b6d18 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:21.347531 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6967d65-4b74-4e81-9298-ab2167753edf None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:21.357645 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6967d65-4b74-4e81-9298-ab2167753edf None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:21.358449 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1381/2775] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:49:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:49:21.365725 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eac9998-3f4a-4a38-afff-176b409dbc8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:21.366962 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac9998-3f4a-4a38-afff-176b409dbc8d admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:21.367232 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac9998-3f4a-4a38-afff-176b409dbc8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:21.367983 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eac9998-3f4a-4a38-afff-176b409dbc8d admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:21.368638 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eac9998-3f4a-4a38-afff-176b409dbc8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:21.371866 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eac9998-3f4a-4a38-afff-176b409dbc8d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:21.455702 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1395/2776] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:49:21 2022] POST /identity/v3/projects => generated 369 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:49:21.459944 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f083c237-3f46-4280-bbbe-d6bcc86b6d18 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:21.478114 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f083c237-3f46-4280-bbbe-d6bcc86b6d18 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:21.665726 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c26f97c-c4d7-4b61-911b-5e691158204c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:21.666810 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c26f97c-c4d7-4b61-911b-5e691158204c admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:21.667294 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c26f97c-c4d7-4b61-911b-5e691158204c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:21.667718 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c26f97c-c4d7-4b61-911b-5e691158204c admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:21.668655 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c26f97c-c4d7-4b61-911b-5e691158204c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:21.671839 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c26f97c-c4d7-4b61-911b-5e691158204c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:21.695113 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:21.719057 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1382/2777] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:49:21 2022] POST /identity/v3/users => generated 285 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:49:21.721495 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eac9998-3f4a-4a38-afff-176b409dbc8d admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:21.750884 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eac9998-3f4a-4a38-afff-176b409dbc8d admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:21.761382 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84382c4-5b94-4f31-8e91-93d7811d45bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:21.762048 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84382c4-5b94-4f31-8e91-93d7811d45bc admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:21.762376 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84382c4-5b94-4f31-8e91-93d7811d45bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:21.762825 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84382c4-5b94-4f31-8e91-93d7811d45bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:21.763648 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84382c4-5b94-4f31-8e91-93d7811d45bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:21.765634 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84382c4-5b94-4f31-8e91-93d7811d45bc admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:21.792573 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1396/2778] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:49:21.794293 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c26f97c-c4d7-4b61-911b-5e691158204c admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:21.807520 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c26f97c-c4d7-4b61-911b-5e691158204c admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:21.815374 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753930d6-72f1-407b-89d2-50bbc473478c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:21.816237 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753930d6-72f1-407b-89d2-50bbc473478c admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:21.816671 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753930d6-72f1-407b-89d2-50bbc473478c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:21.817122 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753930d6-72f1-407b-89d2-50bbc473478c admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:21.822978 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753930d6-72f1-407b-89d2-50bbc473478c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:21.825502 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753930d6-72f1-407b-89d2-50bbc473478c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:21.847852 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1383/2779] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:49:21.852174 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84382c4-5b94-4f31-8e91-93d7811d45bc admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:21.866602 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84382c4-5b94-4f31-8e91-93d7811d45bc admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:21.871825 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d5596-e1af-4955-b9c8-da7b21fe1ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:21.872696 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d5596-e1af-4955-b9c8-da7b21fe1ebd admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:21.873437 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d5596-e1af-4955-b9c8-da7b21fe1ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:21.873917 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340d5596-e1af-4955-b9c8-da7b21fe1ebd admin admin] PATH_INFO: `/v3/projects/2fc81d77f9ac4081a6d7171d53bc2fca/users/5da544334747451c93eb8b7727093e06/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:21.894873 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340d5596-e1af-4955-b9c8-da7b21fe1ebd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fc81d77f9ac4081a6d7171d53bc2fca, user_id=5da544334747451c93eb8b7727093e06, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:21.905814 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340d5596-e1af-4955-b9c8-da7b21fe1ebd admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:21.953133 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1dee8-0f1c-4ecb-9310-fa86e5d18a66 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:21.953490 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1dee8-0f1c-4ecb-9310-fa86e5d18a66 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:21.953712 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1dee8-0f1c-4ecb-9310-fa86e5d18a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:21.958648 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1397/2780] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:49:21 2022] PUT /identity/v3/projects/2fc81d77f9ac4081a6d7171d53bc2fca/users/5da544334747451c93eb8b7727093e06/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:49:22.010301 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-f9e1dee8-0f1c-4ecb-9310-fa86e5d18a66 None None] MFA Rules not processed for user `5da544334747451c93eb8b7727093e06`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:22.013179 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-f9e1dee8-0f1c-4ecb-9310-fa86e5d18a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:22.183470 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1384/2781] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:49:21 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:49:22.188911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d5596-e1af-4955-b9c8-da7b21fe1ebd admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:22.365537 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340d5596-e1af-4955-b9c8-da7b21fe1ebd admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:22.373442 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c0fd5f-5022-43d3-92be-bb42c0f7fae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:22.376617 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0fd5f-5022-43d3-92be-bb42c0f7fae2 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:22.376617 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0fd5f-5022-43d3-92be-bb42c0f7fae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:22.376984 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c0fd5f-5022-43d3-92be-bb42c0f7fae2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:22.380097 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c0fd5f-5022-43d3-92be-bb42c0f7fae2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:22.385700 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c0fd5f-5022-43d3-92be-bb42c0f7fae2 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:22.444845 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1398/2782] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:49:22 2022] POST /identity/v3/projects => generated 367 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:49:22.445994 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e1dee8-0f1c-4ecb-9310-fa86e5d18a66 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:22.473081 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e1dee8-0f1c-4ecb-9310-fa86e5d18a66 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:22.622257 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00748d9e-4f6b-4d60-b5b6-79ceed6a0fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:22.623341 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00748d9e-4f6b-4d60-b5b6-79ceed6a0fb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:22.623791 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00748d9e-4f6b-4d60-b5b6-79ceed6a0fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:22.624253 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00748d9e-4f6b-4d60-b5b6-79ceed6a0fb9 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:22.625175 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00748d9e-4f6b-4d60-b5b6-79ceed6a0fb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:22.627923 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00748d9e-4f6b-4d60-b5b6-79ceed6a0fb9 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:22.690867 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1385/2783] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:49:22 2022] POST /identity/v3/users => generated 290 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:49:22.694901 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c0fd5f-5022-43d3-92be-bb42c0f7fae2 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:22.708091 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c0fd5f-5022-43d3-92be-bb42c0f7fae2 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:22.714978 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42962b2-06b4-480b-8e2f-96708cc3cfea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:22.715764 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42962b2-06b4-480b-8e2f-96708cc3cfea admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:22.716055 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42962b2-06b4-480b-8e2f-96708cc3cfea admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:22.716415 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42962b2-06b4-480b-8e2f-96708cc3cfea admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:22.717072 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42962b2-06b4-480b-8e2f-96708cc3cfea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:22.719114 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42962b2-06b4-480b-8e2f-96708cc3cfea admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:22.751807 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1399/2784] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:49:22.756411 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00748d9e-4f6b-4d60-b5b6-79ceed6a0fb9 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:22.773087 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00748d9e-4f6b-4d60-b5b6-79ceed6a0fb9 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:22.792430 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d859ba75-091a-4cca-81fd-b627e347a489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:22.793198 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d859ba75-091a-4cca-81fd-b627e347a489 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:22.793482 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d859ba75-091a-4cca-81fd-b627e347a489 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:22.793762 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d859ba75-091a-4cca-81fd-b627e347a489 admin admin] PATH_INFO: `/v3/projects/f81aabd0466340d7927707fb76d0172b/users/2715da65af4546f3aa5939742d9edbda/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:22.835300 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d859ba75-091a-4cca-81fd-b627e347a489 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f81aabd0466340d7927707fb76d0172b, user_id=2715da65af4546f3aa5939742d9edbda, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:22.838404 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d859ba75-091a-4cca-81fd-b627e347a489 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:22.887211 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1386/2785] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:49:22 2022] PUT /identity/v3/projects/f81aabd0466340d7927707fb76d0172b/users/2715da65af4546f3aa5939742d9edbda/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:49:22.891215 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42962b2-06b4-480b-8e2f-96708cc3cfea admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:22.905809 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42962b2-06b4-480b-8e2f-96708cc3cfea admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:23.060632 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83bbaee-d945-4486-aaf6-42d29043fc69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:23.061371 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83bbaee-d945-4486-aaf6-42d29043fc69 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:23.062407 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83bbaee-d945-4486-aaf6-42d29043fc69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:23.062726 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83bbaee-d945-4486-aaf6-42d29043fc69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:23.063384 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83bbaee-d945-4486-aaf6-42d29043fc69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:23.066023 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83bbaee-d945-4486-aaf6-42d29043fc69 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:23.093153 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1400/2786] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:49:23.097920 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d859ba75-091a-4cca-81fd-b627e347a489 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:23.119297 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d859ba75-091a-4cca-81fd-b627e347a489 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:23.138618 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7ec600-79b0-4714-b854-bdc3af9a5ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:23.142492 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ec600-79b0-4714-b854-bdc3af9a5ca1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:23.142666 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ec600-79b0-4714-b854-bdc3af9a5ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:23.142837 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ec600-79b0-4714-b854-bdc3af9a5ca1 admin admin] PATH_INFO: `/v3/projects/f81aabd0466340d7927707fb76d0172b/users/2715da65af4546f3aa5939742d9edbda/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:23.156945 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7ec600-79b0-4714-b854-bdc3af9a5ca1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f81aabd0466340d7927707fb76d0172b, user_id=2715da65af4546f3aa5939742d9edbda, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:23.176205 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7ec600-79b0-4714-b854-bdc3af9a5ca1 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:23.238405 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f7a14-704f-4dd4-9ff7-3775786e11dd None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:23.238627 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1387/2787] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:49:23 2022] PUT /identity/v3/projects/f81aabd0466340d7927707fb76d0172b/users/2715da65af4546f3aa5939742d9edbda/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:49:23.238760 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f7a14-704f-4dd4-9ff7-3775786e11dd None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:23.238954 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177f7a14-704f-4dd4-9ff7-3775786e11dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:23.279054 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:23.328184 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-177f7a14-704f-4dd4-9ff7-3775786e11dd None None] MFA Rules not processed for user `2715da65af4546f3aa5939742d9edbda`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:23.329566 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-177f7a14-704f-4dd4-9ff7-3775786e11dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:23.485335 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1401/2788] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:49:23 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:49:23.504715 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7ec600-79b0-4714-b854-bdc3af9a5ca1 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:23.506453 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-aa7ec600-79b0-4714-b854-bdc3af9a5ca1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:23.595863 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:23.661650 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7ec600-79b0-4714-b854-bdc3af9a5ca1 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:23.669879 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cff67d-abb9-4e10-8165-83bb8a4899c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:23.670729 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cff67d-abb9-4e10-8165-83bb8a4899c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:23.670876 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cff67d-abb9-4e10-8165-83bb8a4899c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:23.671148 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cff67d-abb9-4e10-8165-83bb8a4899c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:23.689221 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cff67d-abb9-4e10-8165-83bb8a4899c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:23.696614 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cff67d-abb9-4e10-8165-83bb8a4899c6 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:23.728954 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1388/2789] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:49:23 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:49:23.803963 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177f7a14-704f-4dd4-9ff7-3775786e11dd None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:23.805477 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-177f7a14-704f-4dd4-9ff7-3775786e11dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:23.826433 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177f7a14-704f-4dd4-9ff7-3775786e11dd None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:23.840050 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62704f41-7b01-433d-819b-dc738b7c07cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:23.840689 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62704f41-7b01-433d-819b-dc738b7c07cd service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:23.840969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62704f41-7b01-433d-819b-dc738b7c07cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:23.841239 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62704f41-7b01-433d-819b-dc738b7c07cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:24.056269 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62704f41-7b01-433d-819b-dc738b7c07cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:24.058246 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62704f41-7b01-433d-819b-dc738b7c07cd service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:24.081883 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1402/2790] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:49:23 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:49:26.149681 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7a3754-151d-4005-987d-8bf1b90c6923 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:26.150323 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7a3754-151d-4005-987d-8bf1b90c6923 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:26.150481 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7a3754-151d-4005-987d-8bf1b90c6923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:26.188635 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-8f7a3754-151d-4005-987d-8bf1b90c6923 None None] MFA Rules not processed for user `5fac9ab7665b4f14936adea96ae22ec4`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:26.191130 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-8f7a3754-151d-4005-987d-8bf1b90c6923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:26.319447 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1389/2791] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:49:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:49:26.324205 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62704f41-7b01-433d-819b-dc738b7c07cd service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:26.326095 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-62704f41-7b01-433d-819b-dc738b7c07cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:26.516530 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62704f41-7b01-433d-819b-dc738b7c07cd service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:26.529075 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edbff00-b535-4ac5-a832-c133c5435552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:26.530994 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edbff00-b535-4ac5-a832-c133c5435552 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:26.531222 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edbff00-b535-4ac5-a832-c133c5435552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:26.531482 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edbff00-b535-4ac5-a832-c133c5435552 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:26.532409 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edbff00-b535-4ac5-a832-c133c5435552 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:26.536442 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edbff00-b535-4ac5-a832-c133c5435552 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:26.592588 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1403/2792] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:49:26 2022] POST /identity/v3/projects => generated 377 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:49:26.597528 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7a3754-151d-4005-987d-8bf1b90c6923 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:26.615036 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7a3754-151d-4005-987d-8bf1b90c6923 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:26.712137 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d33297-3c52-430f-a543-93f8690cea57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:26.712808 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d33297-3c52-430f-a543-93f8690cea57 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:26.713084 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d33297-3c52-430f-a543-93f8690cea57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:26.713350 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d33297-3c52-430f-a543-93f8690cea57 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:26.714109 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d33297-3c52-430f-a543-93f8690cea57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:26.716293 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d33297-3c52-430f-a543-93f8690cea57 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:26.746883 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1390/2793] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:49:26 2022] POST /identity/v3/users => generated 295 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:49:26.751191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edbff00-b535-4ac5-a832-c133c5435552 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:26.763364 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edbff00-b535-4ac5-a832-c133c5435552 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:26.769574 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39118d1f-3844-4479-8d06-bed46a43e18b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:26.770643 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39118d1f-3844-4479-8d06-bed46a43e18b admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:26.770972 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39118d1f-3844-4479-8d06-bed46a43e18b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:26.771477 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39118d1f-3844-4479-8d06-bed46a43e18b admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:26.772206 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39118d1f-3844-4479-8d06-bed46a43e18b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:26.774528 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39118d1f-3844-4479-8d06-bed46a43e18b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:26.789903 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1404/2794] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:49:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:49:26.792370 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d33297-3c52-430f-a543-93f8690cea57 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:26.804919 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d33297-3c52-430f-a543-93f8690cea57 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:26.810109 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6890ce71-7a9d-489f-9558-0d3e6ff61bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:26.810982 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890ce71-7a9d-489f-9558-0d3e6ff61bd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:26.811557 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890ce71-7a9d-489f-9558-0d3e6ff61bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:26.812050 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6890ce71-7a9d-489f-9558-0d3e6ff61bd7 admin admin] PATH_INFO: `/v3/projects/9f00f264edf140dba0b727a459c9aada/users/021ed1577ed84746b17a68da8ffd6bea/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:26.835148 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6890ce71-7a9d-489f-9558-0d3e6ff61bd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f00f264edf140dba0b727a459c9aada, user_id=021ed1577ed84746b17a68da8ffd6bea, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:26.837826 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6890ce71-7a9d-489f-9558-0d3e6ff61bd7 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:26.860127 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1391/2795] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:49:26 2022] PUT /identity/v3/projects/9f00f264edf140dba0b727a459c9aada/users/021ed1577ed84746b17a68da8ffd6bea/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:49:26.865103 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39118d1f-3844-4479-8d06-bed46a43e18b admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:26.874255 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:26.880591 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39118d1f-3844-4479-8d06-bed46a43e18b admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:26.966849 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99014e2a-6ecf-4fc1-8b37-bfceb3d04446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:26.967487 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99014e2a-6ecf-4fc1-8b37-bfceb3d04446 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:26.967764 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99014e2a-6ecf-4fc1-8b37-bfceb3d04446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:26.968032 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99014e2a-6ecf-4fc1-8b37-bfceb3d04446 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:26.968669 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99014e2a-6ecf-4fc1-8b37-bfceb3d04446 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:26.970510 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99014e2a-6ecf-4fc1-8b37-bfceb3d04446 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:26.983936 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1405/2796] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:49:26 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:49:26.988463 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6890ce71-7a9d-489f-9558-0d3e6ff61bd7 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:26.999976 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6890ce71-7a9d-489f-9558-0d3e6ff61bd7 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:27.010955 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e5a7b6-380e-4810-b632-033560f05a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:27.011634 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e5a7b6-380e-4810-b632-033560f05a2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:27.011910 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e5a7b6-380e-4810-b632-033560f05a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:27.012185 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e5a7b6-380e-4810-b632-033560f05a2b admin admin] PATH_INFO: `/v3/projects/9f00f264edf140dba0b727a459c9aada/users/021ed1577ed84746b17a68da8ffd6bea/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:27.014243 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e5a7b6-380e-4810-b632-033560f05a2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f00f264edf140dba0b727a459c9aada, user_id=021ed1577ed84746b17a68da8ffd6bea, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:27.017095 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e5a7b6-380e-4810-b632-033560f05a2b admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:27.028619 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1392/2797] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:49:26 2022] PUT /identity/v3/projects/9f00f264edf140dba0b727a459c9aada/users/021ed1577ed84746b17a68da8ffd6bea/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:49:28.583755 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99014e2a-6ecf-4fc1-8b37-bfceb3d04446 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:28.600324 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99014e2a-6ecf-4fc1-8b37-bfceb3d04446 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:28.725894 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd98e33-aebd-432f-af18-ad4e5b202d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:28.726557 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd98e33-aebd-432f-af18-ad4e5b202d04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:28.726852 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd98e33-aebd-432f-af18-ad4e5b202d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:28.727131 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd98e33-aebd-432f-af18-ad4e5b202d04 admin admin] PATH_INFO: `/v3/users/5da544334747451c93eb8b7727093e06` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:28.728505 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd98e33-aebd-432f-af18-ad4e5b202d04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5da544334747451c93eb8b7727093e06)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:28.731813 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd98e33-aebd-432f-af18-ad4e5b202d04 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:28.823208 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6fd98e33-aebd-432f-af18-ad4e5b202d04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5da544334747451c93eb8b7727093e06'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:28.824014 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6fd98e33-aebd-432f-af18-ad4e5b202d04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5da544334747451c93eb8b7727093e06'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:28.841912 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6fd98e33-aebd-432f-af18-ad4e5b202d04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5da544334747451c93eb8b7727093e06'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:28.871285 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6fd98e33-aebd-432f-af18-ad4e5b202d04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5da544334747451c93eb8b7727093e06'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:28.900371 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-6fd98e33-aebd-432f-af18-ad4e5b202d04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5da544334747451c93eb8b7727093e06'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:28.919994 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1406/2798] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:49:28 2022] DELETE /identity/v3/users/5da544334747451c93eb8b7727093e06 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:49:28.925110 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e131147-4bae-4ddd-a261-6185f006b57e None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:28.925622 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e131147-4bae-4ddd-a261-6185f006b57e None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:28.925882 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e131147-4bae-4ddd-a261-6185f006b57e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:29.010090 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-0e131147-4bae-4ddd-a261-6185f006b57e None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:29.011262 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-0e131147-4bae-4ddd-a261-6185f006b57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:29.026738 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:29.131697 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1393/2799] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:49:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:49:29.144905 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd98e33-aebd-432f-af18-ad4e5b202d04 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:29.146608 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6fd98e33-aebd-432f-af18-ad4e5b202d04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:29.190367 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:29.236018 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd98e33-aebd-432f-af18-ad4e5b202d04 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:29.240220 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe28e04-e8f2-4f52-ad4d-09094c7e5bb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:29.240955 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe28e04-e8f2-4f52-ad4d-09094c7e5bb1 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:29.241360 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe28e04-e8f2-4f52-ad4d-09094c7e5bb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:29.241701 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe28e04-e8f2-4f52-ad4d-09094c7e5bb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:29.261018 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe28e04-e8f2-4f52-ad4d-09094c7e5bb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:29.262693 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe28e04-e8f2-4f52-ad4d-09094c7e5bb1 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:29.286175 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1407/2800] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:49:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:49:29.809580 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e131147-4bae-4ddd-a261-6185f006b57e None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:29.811133 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-0e131147-4bae-4ddd-a261-6185f006b57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:30.045189 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e131147-4bae-4ddd-a261-6185f006b57e None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:30.055702 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1604863-54eb-4897-9d37-0cd65f3e71bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:30.056900 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1604863-54eb-4897-9d37-0cd65f3e71bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:30.057348 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1604863-54eb-4897-9d37-0cd65f3e71bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:30.057783 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1604863-54eb-4897-9d37-0cd65f3e71bc admin admin] PATH_INFO: `/v3/projects/2fc81d77f9ac4081a6d7171d53bc2fca` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:30.059603 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1604863-54eb-4897-9d37-0cd65f3e71bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fc81d77f9ac4081a6d7171d53bc2fca)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:30.061831 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1604863-54eb-4897-9d37-0cd65f3e71bc admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:30.202207 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d1604863-54eb-4897-9d37-0cd65f3e71bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fc81d77f9ac4081a6d7171d53bc2fca', 'request_id': 'req-d1604863-54eb-4897-9d37-0cd65f3e71bc'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:30.212997 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d1604863-54eb-4897-9d37-0cd65f3e71bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fc81d77f9ac4081a6d7171d53bc2fca', 'request_id': 'req-d1604863-54eb-4897-9d37-0cd65f3e71bc'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:30.229315 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d1604863-54eb-4897-9d37-0cd65f3e71bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fc81d77f9ac4081a6d7171d53bc2fca', 'request_id': 'req-d1604863-54eb-4897-9d37-0cd65f3e71bc'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:30.256465 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d1604863-54eb-4897-9d37-0cd65f3e71bc admin admin] The token cache is being invalidate because project 2fc81d77f9ac4081a6d7171d53bc2fca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:49:30.256630 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d1604863-54eb-4897-9d37-0cd65f3e71bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:30.257806 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-d1604863-54eb-4897-9d37-0cd65f3e71bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:30.264989 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe28e04-e8f2-4f52-ad4d-09094c7e5bb1 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:30.293509 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1394/2801] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:49:29 2022] DELETE /identity/v3/projects/2fc81d77f9ac4081a6d7171d53bc2fca => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:49:30.333400 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe28e04-e8f2-4f52-ad4d-09094c7e5bb1 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:30.412955 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc467743-4290-4ecc-865d-2e5db51f3f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:30.414056 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc467743-4290-4ecc-865d-2e5db51f3f2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:30.414529 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc467743-4290-4ecc-865d-2e5db51f3f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:30.415140 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc467743-4290-4ecc-865d-2e5db51f3f2a admin admin] PATH_INFO: `/v3/users/2715da65af4546f3aa5939742d9edbda` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:30.416559 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc467743-4290-4ecc-865d-2e5db51f3f2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2715da65af4546f3aa5939742d9edbda)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:30.419652 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc467743-4290-4ecc-865d-2e5db51f3f2a admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:30.539576 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-dc467743-4290-4ecc-865d-2e5db51f3f2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2715da65af4546f3aa5939742d9edbda'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:30.540338 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-dc467743-4290-4ecc-865d-2e5db51f3f2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2715da65af4546f3aa5939742d9edbda'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:30.563067 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-dc467743-4290-4ecc-865d-2e5db51f3f2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2715da65af4546f3aa5939742d9edbda'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:30.588697 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-dc467743-4290-4ecc-865d-2e5db51f3f2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2715da65af4546f3aa5939742d9edbda'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:30.606423 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-dc467743-4290-4ecc-865d-2e5db51f3f2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2715da65af4546f3aa5939742d9edbda'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:30.620313 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1408/2802] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:49:30 2022] DELETE /identity/v3/users/2715da65af4546f3aa5939742d9edbda => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:49:31.231083 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1604863-54eb-4897-9d37-0cd65f3e71bc admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:31.232712 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d1604863-54eb-4897-9d37-0cd65f3e71bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:31.284967 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1604863-54eb-4897-9d37-0cd65f3e71bc admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:31.387608 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:31.436477 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc51e25f-e507-4649-8927-da7c05434cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:31.437146 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc51e25f-e507-4649-8927-da7c05434cb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:31.437422 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc51e25f-e507-4649-8927-da7c05434cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:31.437693 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc51e25f-e507-4649-8927-da7c05434cb1 admin admin] PATH_INFO: `/v3/projects/f81aabd0466340d7927707fb76d0172b` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:31.448842 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc51e25f-e507-4649-8927-da7c05434cb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f81aabd0466340d7927707fb76d0172b)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:31.451500 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc51e25f-e507-4649-8927-da7c05434cb1 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:31.574513 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-bc51e25f-e507-4649-8927-da7c05434cb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f81aabd0466340d7927707fb76d0172b', 'request_id': 'req-bc51e25f-e507-4649-8927-da7c05434cb1'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:31.598477 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-bc51e25f-e507-4649-8927-da7c05434cb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f81aabd0466340d7927707fb76d0172b', 'request_id': 'req-bc51e25f-e507-4649-8927-da7c05434cb1'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:31.609263 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-bc51e25f-e507-4649-8927-da7c05434cb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f81aabd0466340d7927707fb76d0172b', 'request_id': 'req-bc51e25f-e507-4649-8927-da7c05434cb1'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:31.647241 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-bc51e25f-e507-4649-8927-da7c05434cb1 admin admin] The token cache is being invalidate because project f81aabd0466340d7927707fb76d0172b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:49:31.647538 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-bc51e25f-e507-4649-8927-da7c05434cb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:31.653698 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-bc51e25f-e507-4649-8927-da7c05434cb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:31.655617 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1395/2803] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:49:31 2022] DELETE /identity/v3/projects/f81aabd0466340d7927707fb76d0172b => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:49:31.661263 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe37d62-b4d3-4ee9-bf9c-94486b63b481 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:31.661803 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe37d62-b4d3-4ee9-bf9c-94486b63b481 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:31.661803 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe37d62-b4d3-4ee9-bf9c-94486b63b481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:31.695229 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:31.813693 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-6fe37d62-b4d3-4ee9-bf9c-94486b63b481 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:31.818581 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6fe37d62-b4d3-4ee9-bf9c-94486b63b481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:31.952768 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1409/2804] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:49:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:49:31.960695 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516cf140-f4f7-467f-9646-e5567b177473 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:31.961146 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516cf140-f4f7-467f-9646-e5567b177473 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:31.961716 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516cf140-f4f7-467f-9646-e5567b177473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:32.004439 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-516cf140-f4f7-467f-9646-e5567b177473 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:32.009518 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-516cf140-f4f7-467f-9646-e5567b177473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:32.030514 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1396/2805] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:49:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:49:32.036040 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe37d62-b4d3-4ee9-bf9c-94486b63b481 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:32.047990 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe37d62-b4d3-4ee9-bf9c-94486b63b481 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:32.066617 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e002f6c-6284-4997-a407-16b51ff8833f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:32.067212 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e002f6c-6284-4997-a407-16b51ff8833f admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:32.067398 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e002f6c-6284-4997-a407-16b51ff8833f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:32.067653 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e002f6c-6284-4997-a407-16b51ff8833f admin admin] PATH_INFO: `/v3/domains` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:32.068300 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e002f6c-6284-4997-a407-16b51ff8833f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81605) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:49:32.068600 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e002f6c-6284-4997-a407-16b51ff8833f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:32.070449 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e002f6c-6284-4997-a407-16b51ff8833f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:32.109365 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1410/2806] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:49:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:49:32.113992 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516cf140-f4f7-467f-9646-e5567b177473 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:32.124685 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516cf140-f4f7-467f-9646-e5567b177473 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:32.132925 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b993c5-70ef-4e4d-8980-dd615b197e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:32.134472 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b993c5-70ef-4e4d-8980-dd615b197e66 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:32.134916 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b993c5-70ef-4e4d-8980-dd615b197e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:32.135343 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b993c5-70ef-4e4d-8980-dd615b197e66 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:32.136135 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b993c5-70ef-4e4d-8980-dd615b197e66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:32.139123 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b993c5-70ef-4e4d-8980-dd615b197e66 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:32.216558 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1397/2807] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:49:32 2022] POST /identity/v3/projects => generated 365 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:49:32.220934 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e002f6c-6284-4997-a407-16b51ff8833f admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:32.236160 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e002f6c-6284-4997-a407-16b51ff8833f admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:32.391584 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b4b67-8da6-488f-944a-6af76271e811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:32.393046 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b4b67-8da6-488f-944a-6af76271e811 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:32.393482 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b4b67-8da6-488f-944a-6af76271e811 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:32.393944 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b4b67-8da6-488f-944a-6af76271e811 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:32.394981 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0b4b67-8da6-488f-944a-6af76271e811 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:32.402634 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0b4b67-8da6-488f-944a-6af76271e811 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:32.457699 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1411/2808] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:49:32 2022] POST /identity/v3/users => generated 283 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:49:32.459241 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b993c5-70ef-4e4d-8980-dd615b197e66 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:32.481276 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b993c5-70ef-4e4d-8980-dd615b197e66 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:32.493708 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9da8e26-deb5-4c07-8665-b0290e421860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:32.494807 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da8e26-deb5-4c07-8665-b0290e421860 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:32.495477 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da8e26-deb5-4c07-8665-b0290e421860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:32.495908 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da8e26-deb5-4c07-8665-b0290e421860 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:32.497536 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9da8e26-deb5-4c07-8665-b0290e421860 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:32.500804 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9da8e26-deb5-4c07-8665-b0290e421860 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:32.535371 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1398/2809] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:49:32 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:49:32.538538 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b4b67-8da6-488f-944a-6af76271e811 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:32.551104 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b4b67-8da6-488f-944a-6af76271e811 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:32.570390 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b06718a-caae-448c-bcd0-bf3ed500b3e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:32.571027 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06718a-caae-448c-bcd0-bf3ed500b3e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:32.571796 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06718a-caae-448c-bcd0-bf3ed500b3e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:32.571796 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06718a-caae-448c-bcd0-bf3ed500b3e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:32.581330 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b06718a-caae-448c-bcd0-bf3ed500b3e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:32.585650 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b06718a-caae-448c-bcd0-bf3ed500b3e2 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:32.626691 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1412/2810] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:49:32 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:49:32.629281 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9da8e26-deb5-4c07-8665-b0290e421860 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:32.645050 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9da8e26-deb5-4c07-8665-b0290e421860 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:32.656050 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c41fc85-87dd-4838-b262-b087ef97c567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:32.660831 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41fc85-87dd-4838-b262-b087ef97c567 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:32.661397 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41fc85-87dd-4838-b262-b087ef97c567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:32.661943 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41fc85-87dd-4838-b262-b087ef97c567 admin admin] PATH_INFO: `/v3/projects/a1fab16d3a7f427f9605f80d3d0860a7/users/869b384b7c4d47c2881e549a9f77ef78/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:32.718400 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c41fc85-87dd-4838-b262-b087ef97c567 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1fab16d3a7f427f9605f80d3d0860a7, user_id=869b384b7c4d47c2881e549a9f77ef78, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:32.730637 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c41fc85-87dd-4838-b262-b087ef97c567 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:32.772314 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1399/2811] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:49:32 2022] PUT /identity/v3/projects/a1fab16d3a7f427f9605f80d3d0860a7/users/869b384b7c4d47c2881e549a9f77ef78/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:49:32.783657 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8054c33-22bf-41f2-8972-c02f966f59a1 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:32.784595 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8054c33-22bf-41f2-8972-c02f966f59a1 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:32.785069 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8054c33-22bf-41f2-8972-c02f966f59a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:32.839645 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-e8054c33-22bf-41f2-8972-c02f966f59a1 None None] MFA Rules not processed for user `869b384b7c4d47c2881e549a9f77ef78`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:32.841311 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e8054c33-22bf-41f2-8972-c02f966f59a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:32.929431 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:32.991686 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1413/2812] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:49:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:49:32.996009 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c41fc85-87dd-4838-b262-b087ef97c567 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:33.127633 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c41fc85-87dd-4838-b262-b087ef97c567 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:33.132978 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b3cf8-7ee5-42dc-b209-e77c914c2552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:33.133615 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b3cf8-7ee5-42dc-b209-e77c914c2552 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:33.133942 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b3cf8-7ee5-42dc-b209-e77c914c2552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:33.134180 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b3cf8-7ee5-42dc-b209-e77c914c2552 admin admin] PATH_INFO: `/v3/projects` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:33.134819 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79b3cf8-7ee5-42dc-b209-e77c914c2552 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:33.136755 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79b3cf8-7ee5-42dc-b209-e77c914c2552 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:33.144222 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:33.198581 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1400/2813] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:49:32 2022] POST /identity/v3/projects => generated 365 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:49:33.203093 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8054c33-22bf-41f2-8972-c02f966f59a1 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:33.227046 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8054c33-22bf-41f2-8972-c02f966f59a1 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:33.488828 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd1a8b1-6a60-413f-8a5a-c8a701a34ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:33.489652 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd1a8b1-6a60-413f-8a5a-c8a701a34ce0 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:33.493550 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd1a8b1-6a60-413f-8a5a-c8a701a34ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:33.494346 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd1a8b1-6a60-413f-8a5a-c8a701a34ce0 admin admin] PATH_INFO: `/v3/users` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:33.495868 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd1a8b1-6a60-413f-8a5a-c8a701a34ce0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:33.502237 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd1a8b1-6a60-413f-8a5a-c8a701a34ce0 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:33.597277 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1414/2814] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:49:33 2022] POST /identity/v3/users => generated 289 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:49:33.602555 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b3cf8-7ee5-42dc-b209-e77c914c2552 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:33.623155 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b3cf8-7ee5-42dc-b209-e77c914c2552 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:33.634026 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d700e7f2-f4d6-4751-86ec-d8866be3a07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:33.636892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d700e7f2-f4d6-4751-86ec-d8866be3a07d admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:33.637700 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d700e7f2-f4d6-4751-86ec-d8866be3a07d admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:33.638166 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d700e7f2-f4d6-4751-86ec-d8866be3a07d admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:33.645950 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d700e7f2-f4d6-4751-86ec-d8866be3a07d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:33.648859 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d700e7f2-f4d6-4751-86ec-d8866be3a07d admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:33.678002 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd1a8b1-6a60-413f-8a5a-c8a701a34ce0 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:33.702287 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd1a8b1-6a60-413f-8a5a-c8a701a34ce0 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:33.703303 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1401/2815] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:49:33.718451 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2516c2-4279-4d26-8261-20ac6b8dc509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:33.719151 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2516c2-4279-4d26-8261-20ac6b8dc509 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:33.719434 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2516c2-4279-4d26-8261-20ac6b8dc509 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:33.719719 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2516c2-4279-4d26-8261-20ac6b8dc509 admin admin] PATH_INFO: `/v3/projects/8056e5286de54a0b862b9fa6339cd52a/users/18aa0d2cb4ed46d3a038c3b81054cd06/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:33.760286 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2516c2-4279-4d26-8261-20ac6b8dc509 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8056e5286de54a0b862b9fa6339cd52a, user_id=18aa0d2cb4ed46d3a038c3b81054cd06, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:33.763668 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2516c2-4279-4d26-8261-20ac6b8dc509 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:33.798308 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1415/2816] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:49:33 2022] PUT /identity/v3/projects/8056e5286de54a0b862b9fa6339cd52a/users/18aa0d2cb4ed46d3a038c3b81054cd06/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:49:33.802461 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d700e7f2-f4d6-4751-86ec-d8866be3a07d admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:33.820875 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d700e7f2-f4d6-4751-86ec-d8866be3a07d admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:33.962563 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b9749-edba-4377-9045-30af7fec5895 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:33.962720 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b9749-edba-4377-9045-30af7fec5895 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:33.962997 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b9749-edba-4377-9045-30af7fec5895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:34.028191 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc965617-412e-41aa-ac6e-87420f269e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:34.028737 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc965617-412e-41aa-ac6e-87420f269e10 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:34.028984 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc965617-412e-41aa-ac6e-87420f269e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:34.029256 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc965617-412e-41aa-ac6e-87420f269e10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:34.033309 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc965617-412e-41aa-ac6e-87420f269e10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:34.034736 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc965617-412e-41aa-ac6e-87420f269e10 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:34.043866 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-042b9749-edba-4377-9045-30af7fec5895 None None] MFA Rules not processed for user `021ed1577ed84746b17a68da8ffd6bea`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:34.045108 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-042b9749-edba-4377-9045-30af7fec5895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:34.055928 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1402/2817] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:49:33 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:49:34.060047 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc965617-412e-41aa-ac6e-87420f269e10 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:34.086344 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc965617-412e-41aa-ac6e-87420f269e10 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:34.102281 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9137245-e7ff-4654-aba6-3aa7fcf6f395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:34.103283 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9137245-e7ff-4654-aba6-3aa7fcf6f395 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:34.103709 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9137245-e7ff-4654-aba6-3aa7fcf6f395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:34.104186 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9137245-e7ff-4654-aba6-3aa7fcf6f395 admin admin] PATH_INFO: `/v3/projects/8056e5286de54a0b862b9fa6339cd52a/users/18aa0d2cb4ed46d3a038c3b81054cd06/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:34.109788 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9137245-e7ff-4654-aba6-3aa7fcf6f395 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8056e5286de54a0b862b9fa6339cd52a, user_id=18aa0d2cb4ed46d3a038c3b81054cd06, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:34.113113 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9137245-e7ff-4654-aba6-3aa7fcf6f395 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:34.145595 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1403/2818] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:49:34 2022] PUT /identity/v3/projects/8056e5286de54a0b862b9fa6339cd52a/users/18aa0d2cb4ed46d3a038c3b81054cd06/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:49:34.158356 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe24e20-e9fc-4404-83f8-1e2e5cfc9197 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:34.158824 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe24e20-e9fc-4404-83f8-1e2e5cfc9197 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:34.159276 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe24e20-e9fc-4404-83f8-1e2e5cfc9197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:34.182616 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1416/2819] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:49:33 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:49:34.196325 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b9749-edba-4377-9045-30af7fec5895 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:34.197955 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-042b9749-edba-4377-9045-30af7fec5895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:34.211458 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-dfe24e20-e9fc-4404-83f8-1e2e5cfc9197 None None] MFA Rules not processed for user `18aa0d2cb4ed46d3a038c3b81054cd06`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:34.213429 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-dfe24e20-e9fc-4404-83f8-1e2e5cfc9197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:34.259736 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:34.348344 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1404/2820] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:49:34 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:49:34.362679 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe24e20-e9fc-4404-83f8-1e2e5cfc9197 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:34.364003 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-dfe24e20-e9fc-4404-83f8-1e2e5cfc9197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:34.463756 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b9749-edba-4377-9045-30af7fec5895 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:34.485761 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:34.516888 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe24e20-e9fc-4404-83f8-1e2e5cfc9197 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:34.521964 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08397be4-afe9-448a-8c45-cbb90339a391 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:34.522848 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08397be4-afe9-448a-8c45-cbb90339a391 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:34.524904 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08397be4-afe9-448a-8c45-cbb90339a391 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:34.525535 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08397be4-afe9-448a-8c45-cbb90339a391 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:34.545480 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08397be4-afe9-448a-8c45-cbb90339a391 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:34.547153 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08397be4-afe9-448a-8c45-cbb90339a391 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:34.589241 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1405/2821] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:49:34 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:49:34.643686 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11150a14-dcda-4550-9186-ab04077c7710 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf14b64bf04b2ba74d2fe1fd41ff71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:34.644487 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11150a14-dcda-4550-9186-ab04077c7710 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:34.644871 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11150a14-dcda-4550-9186-ab04077c7710 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:34.645242 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11150a14-dcda-4550-9186-ab04077c7710 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:34.647381 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-11150a14-dcda-4550-9186-ab04077c7710 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:34.747671 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08397be4-afe9-448a-8c45-cbb90339a391 service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:34.757726 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11150a14-dcda-4550-9186-ab04077c7710 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:34.759716 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11150a14-dcda-4550-9186-ab04077c7710 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:34.763168 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08397be4-afe9-448a-8c45-cbb90339a391 service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:34.770279 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99818ba4-27d8-40b6-9e43-de428dd69e7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:34.771028 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99818ba4-27d8-40b6-9e43-de428dd69e7c service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:34.773515 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99818ba4-27d8-40b6-9e43-de428dd69e7c service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:34.773823 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99818ba4-27d8-40b6-9e43-de428dd69e7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:34.782303 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1417/2822] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:49:34 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:49:34.891548 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99818ba4-27d8-40b6-9e43-de428dd69e7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:34.893295 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99818ba4-27d8-40b6-9e43-de428dd69e7c service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:34.906673 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1406/2823] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:49:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:49:37.767472 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64d8733-3baa-43c6-a5c8-f78f70e9edf3 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:37.768029 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64d8733-3baa-43c6-a5c8-f78f70e9edf3 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:37.768029 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64d8733-3baa-43c6-a5c8-f78f70e9edf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:37.788140 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-e64d8733-3baa-43c6-a5c8-f78f70e9edf3 None None] MFA Rules not processed for user `a7a959e57e0c4713a855844645fdc26a`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:37.789358 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e64d8733-3baa-43c6-a5c8-f78f70e9edf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:37.803277 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:37.852918 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1418/2824] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:49:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:49:40.102364 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3303472b-af22-411b-bbd9-6b769fef0b9d None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:40.102903 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3303472b-af22-411b-bbd9-6b769fef0b9d None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:40.102903 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3303472b-af22-411b-bbd9-6b769fef0b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:40.139015 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-3303472b-af22-411b-bbd9-6b769fef0b9d None None] MFA Rules not processed for user `c81e5fb007614cf2992f5e02e85d6307`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:40.140230 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3303472b-af22-411b-bbd9-6b769fef0b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:40.207551 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1407/2825] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Sep 28 15:49:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:49:40.222208 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64d8733-3baa-43c6-a5c8-f78f70e9edf3 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:40.223676 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e64d8733-3baa-43c6-a5c8-f78f70e9edf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:40.333646 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64d8733-3baa-43c6-a5c8-f78f70e9edf3 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:40.341471 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737e3942-70ff-4408-a402-5108665c98ce service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '328051fe7607464f8f5dc0dcc01a817a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:40.342276 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e3942-70ff-4408-a402-5108665c98ce service swift] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:40.342656 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e3942-70ff-4408-a402-5108665c98ce service swift] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:40.343043 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e3942-70ff-4408-a402-5108665c98ce service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:40.352828 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737e3942-70ff-4408-a402-5108665c98ce service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:40.354600 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737e3942-70ff-4408-a402-5108665c98ce service swift] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:40.365126 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1419/2826] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 15:49:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 28 15:49:40.730402 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3303472b-af22-411b-bbd9-6b769fef0b9d None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:40.891362 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3303472b-af22-411b-bbd9-6b769fef0b9d None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:40.906260 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e3a11-9463-401f-9b52-92eefb729e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:40.907152 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e3a11-9463-401f-9b52-92eefb729e79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:40.907415 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e3a11-9463-401f-9b52-92eefb729e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:40.907685 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e3a11-9463-401f-9b52-92eefb729e79 admin admin] PATH_INFO: `/v3/users/869b384b7c4d47c2881e549a9f77ef78` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:40.910934 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8e3a11-9463-401f-9b52-92eefb729e79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=869b384b7c4d47c2881e549a9f77ef78)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:40.914633 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8e3a11-9463-401f-9b52-92eefb729e79 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:40.957301 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:41.011713 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3c8e3a11-9463-401f-9b52-92eefb729e79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '869b384b7c4d47c2881e549a9f77ef78'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:41.023912 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3c8e3a11-9463-401f-9b52-92eefb729e79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '869b384b7c4d47c2881e549a9f77ef78'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:41.024809 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3c8e3a11-9463-401f-9b52-92eefb729e79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '869b384b7c4d47c2881e549a9f77ef78'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:41.044530 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3c8e3a11-9463-401f-9b52-92eefb729e79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '869b384b7c4d47c2881e549a9f77ef78'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:41.045238 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3c8e3a11-9463-401f-9b52-92eefb729e79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '869b384b7c4d47c2881e549a9f77ef78'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:41.109028 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1408/2827] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:49:40 2022] DELETE /identity/v3/users/869b384b7c4d47c2881e549a9f77ef78 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:49:41.113957 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c1643-3523-4a96-8416-1e964d5a1bbd None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:41.114343 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c1643-3523-4a96-8416-1e964d5a1bbd None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:41.114799 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c1643-3523-4a96-8416-1e964d5a1bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:41.171372 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-8b0c1643-3523-4a96-8416-1e964d5a1bbd None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:41.175314 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8b0c1643-3523-4a96-8416-1e964d5a1bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:41.374912 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1420/2828] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:49:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:49:41.382785 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e3a11-9463-401f-9b52-92eefb729e79 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:41.384522 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-3c8e3a11-9463-401f-9b52-92eefb729e79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:41.498374 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e3a11-9463-401f-9b52-92eefb729e79 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:41.504158 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca613ff0-285d-4bca-bd24-06a25294c258 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:41.505201 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca613ff0-285d-4bca-bd24-06a25294c258 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:41.505502 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca613ff0-285d-4bca-bd24-06a25294c258 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:41.505794 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca613ff0-285d-4bca-bd24-06a25294c258 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:41.523173 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca613ff0-285d-4bca-bd24-06a25294c258 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:41.524643 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca613ff0-285d-4bca-bd24-06a25294c258 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:41.552190 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1409/2829] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:49:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:49:42.003207 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c1643-3523-4a96-8416-1e964d5a1bbd None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:42.005270 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8b0c1643-3523-4a96-8416-1e964d5a1bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:42.040336 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c1643-3523-4a96-8416-1e964d5a1bbd None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:42.049425 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595d124f-135c-4ce4-b498-ae9fd335dbab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:42.050519 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d124f-135c-4ce4-b498-ae9fd335dbab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:42.051638 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d124f-135c-4ce4-b498-ae9fd335dbab admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:42.052253 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d124f-135c-4ce4-b498-ae9fd335dbab admin admin] PATH_INFO: `/v3/projects/a1fab16d3a7f427f9605f80d3d0860a7` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:42.054787 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595d124f-135c-4ce4-b498-ae9fd335dbab admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1fab16d3a7f427f9605f80d3d0860a7)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:42.064909 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595d124f-135c-4ce4-b498-ae9fd335dbab admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:42.152488 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:42.155119 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-595d124f-135c-4ce4-b498-ae9fd335dbab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1fab16d3a7f427f9605f80d3d0860a7', 'request_id': 'req-595d124f-135c-4ce4-b498-ae9fd335dbab'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:42.176969 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-595d124f-135c-4ce4-b498-ae9fd335dbab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1fab16d3a7f427f9605f80d3d0860a7', 'request_id': 'req-595d124f-135c-4ce4-b498-ae9fd335dbab'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:42.187816 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-595d124f-135c-4ce4-b498-ae9fd335dbab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1fab16d3a7f427f9605f80d3d0860a7', 'request_id': 'req-595d124f-135c-4ce4-b498-ae9fd335dbab'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:42.201023 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-595d124f-135c-4ce4-b498-ae9fd335dbab admin admin] The token cache is being invalidate because project a1fab16d3a7f427f9605f80d3d0860a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:49:42.201653 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-595d124f-135c-4ce4-b498-ae9fd335dbab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:42.203091 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-595d124f-135c-4ce4-b498-ae9fd335dbab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:42.205738 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1421/2830] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:49:42 2022] DELETE /identity/v3/projects/a1fab16d3a7f427f9605f80d3d0860a7 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:49:42.209791 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca613ff0-285d-4bca-bd24-06a25294c258 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:42.228672 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca613ff0-285d-4bca-bd24-06a25294c258 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:42.348694 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e092838-07e3-4472-a318-c16d1d4e6fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:42.350327 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e092838-07e3-4472-a318-c16d1d4e6fbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:42.350993 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e092838-07e3-4472-a318-c16d1d4e6fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:42.351785 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e092838-07e3-4472-a318-c16d1d4e6fbe admin admin] PATH_INFO: `/v3/users/18aa0d2cb4ed46d3a038c3b81054cd06` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:42.354630 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e092838-07e3-4472-a318-c16d1d4e6fbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=18aa0d2cb4ed46d3a038c3b81054cd06)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:42.358482 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e092838-07e3-4472-a318-c16d1d4e6fbe admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:42.452479 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:42.455058 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3e092838-07e3-4472-a318-c16d1d4e6fbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18aa0d2cb4ed46d3a038c3b81054cd06'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:42.465431 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3e092838-07e3-4472-a318-c16d1d4e6fbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18aa0d2cb4ed46d3a038c3b81054cd06'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:42.466852 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3e092838-07e3-4472-a318-c16d1d4e6fbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18aa0d2cb4ed46d3a038c3b81054cd06'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:42.494263 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3e092838-07e3-4472-a318-c16d1d4e6fbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18aa0d2cb4ed46d3a038c3b81054cd06'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:42.495057 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-3e092838-07e3-4472-a318-c16d1d4e6fbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18aa0d2cb4ed46d3a038c3b81054cd06'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:42.523784 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1410/2831] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:49:42 2022] DELETE /identity/v3/users/18aa0d2cb4ed46d3a038c3b81054cd06 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:49:42.964192 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595d124f-135c-4ce4-b498-ae9fd335dbab admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:42.965675 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-595d124f-135c-4ce4-b498-ae9fd335dbab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:42.986606 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595d124f-135c-4ce4-b498-ae9fd335dbab admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:43.080110 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53526f1c-1cee-4470-a0cb-b2944f649a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:43.081142 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53526f1c-1cee-4470-a0cb-b2944f649a0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:43.081618 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53526f1c-1cee-4470-a0cb-b2944f649a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:43.082049 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53526f1c-1cee-4470-a0cb-b2944f649a0b admin admin] PATH_INFO: `/v3/projects/8056e5286de54a0b862b9fa6339cd52a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:43.083352 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53526f1c-1cee-4470-a0cb-b2944f649a0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8056e5286de54a0b862b9fa6339cd52a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:43.085488 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53526f1c-1cee-4470-a0cb-b2944f649a0b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:43.148016 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-53526f1c-1cee-4470-a0cb-b2944f649a0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8056e5286de54a0b862b9fa6339cd52a', 'request_id': 'req-53526f1c-1cee-4470-a0cb-b2944f649a0b'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:43.162621 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-53526f1c-1cee-4470-a0cb-b2944f649a0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8056e5286de54a0b862b9fa6339cd52a', 'request_id': 'req-53526f1c-1cee-4470-a0cb-b2944f649a0b'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:43.173763 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-53526f1c-1cee-4470-a0cb-b2944f649a0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8056e5286de54a0b862b9fa6339cd52a', 'request_id': 'req-53526f1c-1cee-4470-a0cb-b2944f649a0b'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:43.185978 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-53526f1c-1cee-4470-a0cb-b2944f649a0b admin admin] The token cache is being invalidate because project 8056e5286de54a0b862b9fa6339cd52a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:49:43.186352 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-53526f1c-1cee-4470-a0cb-b2944f649a0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:43.186970 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-53526f1c-1cee-4470-a0cb-b2944f649a0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:43.188708 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1422/2832] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:49:42 2022] DELETE /identity/v3/projects/8056e5286de54a0b862b9fa6339cd52a => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:49:43.193478 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b390b2f-8a11-4bd4-b881-5f0e09d1cd0c None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:43.193840 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b390b2f-8a11-4bd4-b881-5f0e09d1cd0c None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:43.194141 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b390b2f-8a11-4bd4-b881-5f0e09d1cd0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:43.228645 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-0b390b2f-8a11-4bd4-b881-5f0e09d1cd0c None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:43.229906 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-0b390b2f-8a11-4bd4-b881-5f0e09d1cd0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:43.322457 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1411/2833] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:49:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:49:43.325412 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc2f92-35f5-41b9-919d-6fe26efd7a5f None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:43.325987 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc2f92-35f5-41b9-919d-6fe26efd7a5f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:43.326446 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc2f92-35f5-41b9-919d-6fe26efd7a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:43.357591 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-fbbc2f92-35f5-41b9-919d-6fe26efd7a5f None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:43.358960 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-fbbc2f92-35f5-41b9-919d-6fe26efd7a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:43.369879 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1423/2834] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:49:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:49:43.372442 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b390b2f-8a11-4bd4-b881-5f0e09d1cd0c None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:43.383453 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b390b2f-8a11-4bd4-b881-5f0e09d1cd0c None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:43.388777 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e22b74e-a7cf-48d3-89b3-7a363f827189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:43.389446 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e22b74e-a7cf-48d3-89b3-7a363f827189 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:43.389751 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e22b74e-a7cf-48d3-89b3-7a363f827189 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:43.390088 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e22b74e-a7cf-48d3-89b3-7a363f827189 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:43.390803 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e22b74e-a7cf-48d3-89b3-7a363f827189 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:49:43.391124 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e22b74e-a7cf-48d3-89b3-7a363f827189 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:43.392967 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e22b74e-a7cf-48d3-89b3-7a363f827189 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:43.406361 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1412/2835] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:49:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 28 15:49:43.410724 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbc2f92-35f5-41b9-919d-6fe26efd7a5f None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:43.420199 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbc2f92-35f5-41b9-919d-6fe26efd7a5f None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:43.425639 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950f194e-5630-4702-baf9-8763df870d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:43.426286 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f194e-5630-4702-baf9-8763df870d2c admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:43.426606 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f194e-5630-4702-baf9-8763df870d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:43.426899 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f194e-5630-4702-baf9-8763df870d2c admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:43.427517 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950f194e-5630-4702-baf9-8763df870d2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:43.429442 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950f194e-5630-4702-baf9-8763df870d2c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:43.459538 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1424/2836] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:49:43 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:49:43.464248 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e22b74e-a7cf-48d3-89b3-7a363f827189 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:43.476662 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e22b74e-a7cf-48d3-89b3-7a363f827189 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:43.567198 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db93f5f3-922a-4ca8-8ad7-b3dade0e70fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:43.567824 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db93f5f3-922a-4ca8-8ad7-b3dade0e70fb admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:43.568099 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db93f5f3-922a-4ca8-8ad7-b3dade0e70fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:43.568364 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db93f5f3-922a-4ca8-8ad7-b3dade0e70fb admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:43.569002 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db93f5f3-922a-4ca8-8ad7-b3dade0e70fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:43.570976 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db93f5f3-922a-4ca8-8ad7-b3dade0e70fb admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:43.601031 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950f194e-5630-4702-baf9-8763df870d2c admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:43.602522 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1413/2837] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:49:43 2022] POST /identity/v3/users => generated 293 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:49:43.614250 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950f194e-5630-4702-baf9-8763df870d2c admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:43.619557 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceab57a-e062-4ff4-bc2f-92902eae26bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:43.620324 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceab57a-e062-4ff4-bc2f-92902eae26bb admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:43.620674 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceab57a-e062-4ff4-bc2f-92902eae26bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:43.620941 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceab57a-e062-4ff4-bc2f-92902eae26bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:43.621584 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceab57a-e062-4ff4-bc2f-92902eae26bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:43.623263 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceab57a-e062-4ff4-bc2f-92902eae26bb admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:43.636889 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1425/2838] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:49:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:49:43.641041 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db93f5f3-922a-4ca8-8ad7-b3dade0e70fb admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:43.651685 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db93f5f3-922a-4ca8-8ad7-b3dade0e70fb admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:43.656358 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7571ba6-8896-4a15-a49d-7671ab0b6c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:43.657556 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7571ba6-8896-4a15-a49d-7671ab0b6c14 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:43.657982 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7571ba6-8896-4a15-a49d-7671ab0b6c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:43.658256 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7571ba6-8896-4a15-a49d-7671ab0b6c14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:43.658903 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7571ba6-8896-4a15-a49d-7671ab0b6c14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:43.660597 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7571ba6-8896-4a15-a49d-7671ab0b6c14 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:43.676310 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1414/2839] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:49:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:49:43.680591 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceab57a-e062-4ff4-bc2f-92902eae26bb admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:43.691546 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceab57a-e062-4ff4-bc2f-92902eae26bb admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:43.696770 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba65831-89cd-4394-8e40-858598a49283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:43.697631 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba65831-89cd-4394-8e40-858598a49283 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:43.698160 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba65831-89cd-4394-8e40-858598a49283 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:43.698594 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba65831-89cd-4394-8e40-858598a49283 admin admin] PATH_INFO: `/v3/projects/f0072803cef147f7911f1a4cccded688/users/03b8f74c3b6e41ccbe6ecd5ecd618ffa/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:43.716167 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba65831-89cd-4394-8e40-858598a49283 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0072803cef147f7911f1a4cccded688, user_id=03b8f74c3b6e41ccbe6ecd5ecd618ffa, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:43.718837 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba65831-89cd-4394-8e40-858598a49283 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:43.735330 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1426/2840] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:49:43 2022] PUT /identity/v3/projects/f0072803cef147f7911f1a4cccded688/users/03b8f74c3b6e41ccbe6ecd5ecd618ffa/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:49:43.747642 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7069b48f-447f-462c-aee2-07c535ea8cdc None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:43.747926 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7069b48f-447f-462c-aee2-07c535ea8cdc None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:43.748199 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7069b48f-447f-462c-aee2-07c535ea8cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:43.787742 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-7069b48f-447f-462c-aee2-07c535ea8cdc None None] MFA Rules not processed for user `03b8f74c3b6e41ccbe6ecd5ecd618ffa`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:43.788990 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7069b48f-447f-462c-aee2-07c535ea8cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:43.812960 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:43.898276 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1415/2841] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:49:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:49:43.915301 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba65831-89cd-4394-8e40-858598a49283 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:43.917077 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-8ba65831-89cd-4394-8e40-858598a49283 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:43.926662 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:44.000146 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba65831-89cd-4394-8e40-858598a49283 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:44.005001 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f295edca-490c-494f-b0b0-4d554f18d356 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:44.005789 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295edca-490c-494f-b0b0-4d554f18d356 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:44.006106 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295edca-490c-494f-b0b0-4d554f18d356 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:44.006389 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295edca-490c-494f-b0b0-4d554f18d356 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:44.015939 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f295edca-490c-494f-b0b0-4d554f18d356 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:44.017985 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f295edca-490c-494f-b0b0-4d554f18d356 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:44.030783 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1427/2842] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:49:43 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:49:52.772496 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7069b48f-447f-462c-aee2-07c535ea8cdc None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:52.833431 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7069b48f-447f-462c-aee2-07c535ea8cdc None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:52.837879 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db88679f-f47a-4e7c-abff-bd9b71fcf926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:52.838504 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db88679f-f47a-4e7c-abff-bd9b71fcf926 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:52.838780 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db88679f-f47a-4e7c-abff-bd9b71fcf926 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:52.839048 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db88679f-f47a-4e7c-abff-bd9b71fcf926 admin admin] PATH_INFO: `/v3/users/03b8f74c3b6e41ccbe6ecd5ecd618ffa` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:52.840166 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db88679f-f47a-4e7c-abff-bd9b71fcf926 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03b8f74c3b6e41ccbe6ecd5ecd618ffa)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:52.842092 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db88679f-f47a-4e7c-abff-bd9b71fcf926 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:52.887435 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-db88679f-f47a-4e7c-abff-bd9b71fcf926 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03b8f74c3b6e41ccbe6ecd5ecd618ffa'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:52.894799 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-db88679f-f47a-4e7c-abff-bd9b71fcf926 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03b8f74c3b6e41ccbe6ecd5ecd618ffa'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:52.895454 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-db88679f-f47a-4e7c-abff-bd9b71fcf926 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03b8f74c3b6e41ccbe6ecd5ecd618ffa'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:52.902502 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-db88679f-f47a-4e7c-abff-bd9b71fcf926 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03b8f74c3b6e41ccbe6ecd5ecd618ffa'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:52.903111 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-db88679f-f47a-4e7c-abff-bd9b71fcf926 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03b8f74c3b6e41ccbe6ecd5ecd618ffa'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:52.920069 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1416/2843] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:49:52 2022] DELETE /identity/v3/users/03b8f74c3b6e41ccbe6ecd5ecd618ffa => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:49:52.924466 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c409dd-28d0-4bfb-8fef-69a4ef714cad None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:52.924979 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c409dd-28d0-4bfb-8fef-69a4ef714cad None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:52.925095 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c409dd-28d0-4bfb-8fef-69a4ef714cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:52.944519 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-59c409dd-28d0-4bfb-8fef-69a4ef714cad None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:52.945705 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-59c409dd-28d0-4bfb-8fef-69a4ef714cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:53.005110 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1428/2844] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:49:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:49:53.014732 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db88679f-f47a-4e7c-abff-bd9b71fcf926 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:53.016110 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-db88679f-f47a-4e7c-abff-bd9b71fcf926 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:53.058476 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:53.096553 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db88679f-f47a-4e7c-abff-bd9b71fcf926 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:53.100227 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6163e10a-b067-43ef-8da2-644956af192e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:53.100827 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163e10a-b067-43ef-8da2-644956af192e service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:53.101104 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163e10a-b067-43ef-8da2-644956af192e service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:53.101376 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163e10a-b067-43ef-8da2-644956af192e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:53.111288 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6163e10a-b067-43ef-8da2-644956af192e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:53.112690 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6163e10a-b067-43ef-8da2-644956af192e service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:53.122641 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1417/2845] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:49:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:49:53.383592 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c409dd-28d0-4bfb-8fef-69a4ef714cad None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:53.385203 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-59c409dd-28d0-4bfb-8fef-69a4ef714cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:53.398537 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c409dd-28d0-4bfb-8fef-69a4ef714cad None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:53.402976 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b2424-018e-430b-9ae2-a3c20d680c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:53.403596 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b2424-018e-430b-9ae2-a3c20d680c3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:53.403875 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b2424-018e-430b-9ae2-a3c20d680c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:53.404141 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b2424-018e-430b-9ae2-a3c20d680c3c admin admin] PATH_INFO: `/v3/projects/f0072803cef147f7911f1a4cccded688` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:53.405263 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3b2424-018e-430b-9ae2-a3c20d680c3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0072803cef147f7911f1a4cccded688)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:53.408091 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3b2424-018e-430b-9ae2-a3c20d680c3c admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:53.465653 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5c3b2424-018e-430b-9ae2-a3c20d680c3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0072803cef147f7911f1a4cccded688', 'request_id': 'req-5c3b2424-018e-430b-9ae2-a3c20d680c3c'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:53.473513 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5c3b2424-018e-430b-9ae2-a3c20d680c3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0072803cef147f7911f1a4cccded688', 'request_id': 'req-5c3b2424-018e-430b-9ae2-a3c20d680c3c'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:53.482752 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5c3b2424-018e-430b-9ae2-a3c20d680c3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0072803cef147f7911f1a4cccded688', 'request_id': 'req-5c3b2424-018e-430b-9ae2-a3c20d680c3c'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:53.496922 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5c3b2424-018e-430b-9ae2-a3c20d680c3c admin admin] The token cache is being invalidate because project f0072803cef147f7911f1a4cccded688 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:49:53.497403 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5c3b2424-018e-430b-9ae2-a3c20d680c3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:53.498218 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-5c3b2424-018e-430b-9ae2-a3c20d680c3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:49:53.500102 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1429/2846] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:49:53 2022] DELETE /identity/v3/projects/f0072803cef147f7911f1a4cccded688 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:49:53.504999 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20596ce-97e7-4ebf-861e-34bcee99f6b6 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:53.505335 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20596ce-97e7-4ebf-861e-34bcee99f6b6 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:53.505556 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20596ce-97e7-4ebf-861e-34bcee99f6b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:53.534614 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-d20596ce-97e7-4ebf-861e-34bcee99f6b6 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:53.535833 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d20596ce-97e7-4ebf-861e-34bcee99f6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:53.615680 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1418/2847] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:49:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:49:53.623073 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2b808-730a-494b-91eb-363e2a7de9a8 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:53.623550 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2b808-730a-494b-91eb-363e2a7de9a8 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:53.623888 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2b808-730a-494b-91eb-363e2a7de9a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:53.645345 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-fdf2b808-730a-494b-91eb-363e2a7de9a8 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:53.646563 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-fdf2b808-730a-494b-91eb-363e2a7de9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:53.654356 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1430/2848] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:49:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:49:53.658587 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20596ce-97e7-4ebf-861e-34bcee99f6b6 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:53.668217 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20596ce-97e7-4ebf-861e-34bcee99f6b6 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:53.672731 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb599390-949f-49e2-aff4-3f8c63f91fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:53.673706 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb599390-949f-49e2-aff4-3f8c63f91fef admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:53.674014 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb599390-949f-49e2-aff4-3f8c63f91fef admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:53.674285 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb599390-949f-49e2-aff4-3f8c63f91fef admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:53.674928 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb599390-949f-49e2-aff4-3f8c63f91fef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:49:53.675227 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb599390-949f-49e2-aff4-3f8c63f91fef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:53.676805 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb599390-949f-49e2-aff4-3f8c63f91fef admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:53.688243 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1419/2849] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:49:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:49:53.692892 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf2b808-730a-494b-91eb-363e2a7de9a8 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:53.701966 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf2b808-730a-494b-91eb-363e2a7de9a8 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:53.707422 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81575ce-acfd-4cdf-8992-beeb0e68fe03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:53.708039 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81575ce-acfd-4cdf-8992-beeb0e68fe03 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:53.708317 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81575ce-acfd-4cdf-8992-beeb0e68fe03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:53.708584 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81575ce-acfd-4cdf-8992-beeb0e68fe03 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:53.709195 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81575ce-acfd-4cdf-8992-beeb0e68fe03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:53.711313 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81575ce-acfd-4cdf-8992-beeb0e68fe03 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:53.733137 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:53.741567 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1431/2850] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:49:53 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:49:53.746743 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb599390-949f-49e2-aff4-3f8c63f91fef admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:53.757503 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb599390-949f-49e2-aff4-3f8c63f91fef admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:53.813246 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30debf39-d031-410d-9a02-207a2c3b75de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:53.814462 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30debf39-d031-410d-9a02-207a2c3b75de admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:53.814804 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30debf39-d031-410d-9a02-207a2c3b75de admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:53.815086 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30debf39-d031-410d-9a02-207a2c3b75de admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:53.815732 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30debf39-d031-410d-9a02-207a2c3b75de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:53.817642 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30debf39-d031-410d-9a02-207a2c3b75de admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:53.840321 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1420/2851] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:49:53 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:49:53.844525 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81575ce-acfd-4cdf-8992-beeb0e68fe03 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:53.854775 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81575ce-acfd-4cdf-8992-beeb0e68fe03 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:53.859594 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ea60ad-e25d-4ba4-8786-101f86ed0b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:53.860203 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea60ad-e25d-4ba4-8786-101f86ed0b6b admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:53.860480 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea60ad-e25d-4ba4-8786-101f86ed0b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:53.861289 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea60ad-e25d-4ba4-8786-101f86ed0b6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:53.862001 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ea60ad-e25d-4ba4-8786-101f86ed0b6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:53.863592 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ea60ad-e25d-4ba4-8786-101f86ed0b6b admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:53.877951 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1432/2852] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:49:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:49:53.879026 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30debf39-d031-410d-9a02-207a2c3b75de admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:53.888991 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30debf39-d031-410d-9a02-207a2c3b75de admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:53.893929 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469f271e-fa5b-4a96-b993-84606674615c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:53.894528 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f271e-fa5b-4a96-b993-84606674615c admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:53.894806 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f271e-fa5b-4a96-b993-84606674615c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:53.895076 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f271e-fa5b-4a96-b993-84606674615c admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:53.895718 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469f271e-fa5b-4a96-b993-84606674615c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:53.897307 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469f271e-fa5b-4a96-b993-84606674615c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:53.908696 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1421/2853] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:49:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:49:53.913072 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ea60ad-e25d-4ba4-8786-101f86ed0b6b admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:53.922847 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ea60ad-e25d-4ba4-8786-101f86ed0b6b admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:53.927983 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77472b66-1c0a-4a1b-bde1-c7301ef60700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:53.928633 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77472b66-1c0a-4a1b-bde1-c7301ef60700 admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:53.928911 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77472b66-1c0a-4a1b-bde1-c7301ef60700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:53.929188 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77472b66-1c0a-4a1b-bde1-c7301ef60700 admin admin] PATH_INFO: `/v3/projects/a0260d8fef7046bb9dac0d4aadc5e401/users/6241921540904e3880e55f7b819a1959/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:53.944596 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77472b66-1c0a-4a1b-bde1-c7301ef60700 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0260d8fef7046bb9dac0d4aadc5e401, user_id=6241921540904e3880e55f7b819a1959, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:53.947365 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77472b66-1c0a-4a1b-bde1-c7301ef60700 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:53.960691 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1433/2854] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:49:53 2022] PUT /identity/v3/projects/a0260d8fef7046bb9dac0d4aadc5e401/users/6241921540904e3880e55f7b819a1959/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:49:53.974542 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f49573-8829-48c5-821f-8a2e889064df None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:53.974822 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f49573-8829-48c5-821f-8a2e889064df None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:53.975110 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f49573-8829-48c5-821f-8a2e889064df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:54.002868 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-c9f49573-8829-48c5-821f-8a2e889064df None None] MFA Rules not processed for user `6241921540904e3880e55f7b819a1959`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:54.004070 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c9f49573-8829-48c5-821f-8a2e889064df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:54.016750 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:54.069017 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1422/2855] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:49:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:49:54.073366 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77472b66-1c0a-4a1b-bde1-c7301ef60700 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:54.134934 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77472b66-1c0a-4a1b-bde1-c7301ef60700 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:54.139591 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1995b61-0ea0-4f51-b3be-1ee43f4c3c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:54.140299 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1995b61-0ea0-4f51-b3be-1ee43f4c3c52 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:54.140459 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1995b61-0ea0-4f51-b3be-1ee43f4c3c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:54.140731 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1995b61-0ea0-4f51-b3be-1ee43f4c3c52 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:54.141354 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1995b61-0ea0-4f51-b3be-1ee43f4c3c52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:54.143769 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1995b61-0ea0-4f51-b3be-1ee43f4c3c52 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:54.170842 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1434/2856] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:49:54 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:49:54.174864 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f49573-8829-48c5-821f-8a2e889064df None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:54.185885 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f49573-8829-48c5-821f-8a2e889064df None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:54.245373 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112ac416-67c8-4bbc-8923-9061573c3f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:54.246016 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ac416-67c8-4bbc-8923-9061573c3f61 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:54.246295 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ac416-67c8-4bbc-8923-9061573c3f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:54.246563 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ac416-67c8-4bbc-8923-9061573c3f61 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:54.247212 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112ac416-67c8-4bbc-8923-9061573c3f61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:54.247451 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203295e-c823-4480-a45e-f0572dc2e841 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:54.247732 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203295e-c823-4480-a45e-f0572dc2e841 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:54.248002 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203295e-c823-4480-a45e-f0572dc2e841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:54.249105 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112ac416-67c8-4bbc-8923-9061573c3f61 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:54.269149 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-7203295e-c823-4480-a45e-f0572dc2e841 None None] MFA Rules not processed for user `5fac9ab7665b4f14936adea96ae22ec4`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:54.270395 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7203295e-c823-4480-a45e-f0572dc2e841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:54.276456 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1423/2857] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:49:54 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:49:54.281234 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112ac416-67c8-4bbc-8923-9061573c3f61 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:54.293560 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112ac416-67c8-4bbc-8923-9061573c3f61 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:54.297400 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb006c-3803-4bc6-9de0-3b9bbbc33486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:54.298409 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb006c-3803-4bc6-9de0-3b9bbbc33486 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:54.298502 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb006c-3803-4bc6-9de0-3b9bbbc33486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:54.298710 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb006c-3803-4bc6-9de0-3b9bbbc33486 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:54.299358 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abb006c-3803-4bc6-9de0-3b9bbbc33486 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:54.301340 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abb006c-3803-4bc6-9de0-3b9bbbc33486 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:54.311506 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1424/2858] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:49:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:49:54.315834 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb006c-3803-4bc6-9de0-3b9bbbc33486 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:54.325405 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb006c-3803-4bc6-9de0-3b9bbbc33486 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:54.330448 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf2fd09-9ad5-452b-afa6-5919fc3193a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:54.331015 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf2fd09-9ad5-452b-afa6-5919fc3193a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:54.331287 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf2fd09-9ad5-452b-afa6-5919fc3193a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:54.331660 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf2fd09-9ad5-452b-afa6-5919fc3193a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:54.332273 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf2fd09-9ad5-452b-afa6-5919fc3193a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:54.333811 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf2fd09-9ad5-452b-afa6-5919fc3193a9 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:54.338404 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1435/2859] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:49:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:49:54.347338 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1425/2860] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:49:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:49:54.348487 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7203295e-c823-4480-a45e-f0572dc2e841 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:54.359201 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7203295e-c823-4480-a45e-f0572dc2e841 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:54.364052 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd42734f-3cad-41de-863e-4337877424cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:54.364965 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42734f-3cad-41de-863e-4337877424cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:54.365245 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42734f-3cad-41de-863e-4337877424cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:54.365522 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42734f-3cad-41de-863e-4337877424cd admin admin] PATH_INFO: `/v3/projects/9f138b87ff3e4ac29678ef3a5ce353c8/users/29f2486a0a484da7a623351baf37876b/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:54.382323 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd42734f-3cad-41de-863e-4337877424cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f138b87ff3e4ac29678ef3a5ce353c8, user_id=29f2486a0a484da7a623351baf37876b, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:54.384989 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd42734f-3cad-41de-863e-4337877424cd admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:54.401565 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1436/2861] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:49:54 2022] PUT /identity/v3/projects/9f138b87ff3e4ac29678ef3a5ce353c8/users/29f2486a0a484da7a623351baf37876b/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:49:54.470883 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1468898c-015c-49fa-9aa9-c9ccea8a31be None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:54.471254 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1468898c-015c-49fa-9aa9-c9ccea8a31be None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:54.471520 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1468898c-015c-49fa-9aa9-c9ccea8a31be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:54.500633 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-1468898c-015c-49fa-9aa9-c9ccea8a31be None None] MFA Rules not processed for user `29f2486a0a484da7a623351baf37876b`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:54.502400 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-1468898c-015c-49fa-9aa9-c9ccea8a31be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:54.569671 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1426/2862] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:49:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:49:54.574130 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd42734f-3cad-41de-863e-4337877424cd admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:54.609907 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:54.645342 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd42734f-3cad-41de-863e-4337877424cd admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:54.650340 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06398293-7c2a-4fba-8ae5-13ea498c2cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:54.650951 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06398293-7c2a-4fba-8ae5-13ea498c2cc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:54.651235 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06398293-7c2a-4fba-8ae5-13ea498c2cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:54.651511 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06398293-7c2a-4fba-8ae5-13ea498c2cc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:54.652141 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06398293-7c2a-4fba-8ae5-13ea498c2cc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:54.654301 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06398293-7c2a-4fba-8ae5-13ea498c2cc1 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:54.686143 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1437/2863] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:49:54 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:49:54.690686 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1468898c-015c-49fa-9aa9-c9ccea8a31be None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:54.701294 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1468898c-015c-49fa-9aa9-c9ccea8a31be None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:54.710238 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:54.762183 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2ffa8b-cb72-4841-85ad-75e9ca5990b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:54.762905 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ffa8b-cb72-4841-85ad-75e9ca5990b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:54.763245 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ffa8b-cb72-4841-85ad-75e9ca5990b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:54.763520 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ffa8b-cb72-4841-85ad-75e9ca5990b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:54.764176 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2ffa8b-cb72-4841-85ad-75e9ca5990b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:54.766149 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2ffa8b-cb72-4841-85ad-75e9ca5990b3 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:54.790762 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1427/2864] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:49:54 2022] POST /identity/v3/users => generated 299 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:49:54.792080 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06398293-7c2a-4fba-8ae5-13ea498c2cc1 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:54.803026 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06398293-7c2a-4fba-8ae5-13ea498c2cc1 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:54.807674 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95316ffc-3620-4b48-b03c-db3c6755b8c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:54.808341 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95316ffc-3620-4b48-b03c-db3c6755b8c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:54.808628 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95316ffc-3620-4b48-b03c-db3c6755b8c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:54.808904 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95316ffc-3620-4b48-b03c-db3c6755b8c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:54.809652 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95316ffc-3620-4b48-b03c-db3c6755b8c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:54.811208 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95316ffc-3620-4b48-b03c-db3c6755b8c5 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:54.822782 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1438/2865] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:49:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:49:54.826944 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2ffa8b-cb72-4841-85ad-75e9ca5990b3 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:54.837963 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2ffa8b-cb72-4841-85ad-75e9ca5990b3 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:54.843277 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b349831-9713-4e59-ba7f-aea527009639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:54.843938 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b349831-9713-4e59-ba7f-aea527009639 admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:54.844220 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b349831-9713-4e59-ba7f-aea527009639 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:54.844495 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b349831-9713-4e59-ba7f-aea527009639 admin admin] PATH_INFO: `/v3/projects/0384852dbc38442cba25983c5abb7126/users/bc9712a3e2a3459c9147883548db8fdc/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:54.861145 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b349831-9713-4e59-ba7f-aea527009639 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0384852dbc38442cba25983c5abb7126, user_id=bc9712a3e2a3459c9147883548db8fdc, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:54.863902 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b349831-9713-4e59-ba7f-aea527009639 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:54.877330 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1428/2866] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:49:54 2022] PUT /identity/v3/projects/0384852dbc38442cba25983c5abb7126/users/bc9712a3e2a3459c9147883548db8fdc/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:49:54.881038 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95316ffc-3620-4b48-b03c-db3c6755b8c5 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:54.893210 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95316ffc-3620-4b48-b03c-db3c6755b8c5 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:54.952829 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6abbd2-d7b4-45fa-a646-589e1866e129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:54.953473 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6abbd2-d7b4-45fa-a646-589e1866e129 admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:54.953715 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6abbd2-d7b4-45fa-a646-589e1866e129 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:54.954023 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6abbd2-d7b4-45fa-a646-589e1866e129 admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:54.955339 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6abbd2-d7b4-45fa-a646-589e1866e129 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:54.956936 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6abbd2-d7b4-45fa-a646-589e1866e129 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:54.967215 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1439/2867] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:49:54 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:49:54.971472 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b349831-9713-4e59-ba7f-aea527009639 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:54.981970 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b349831-9713-4e59-ba7f-aea527009639 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:54.987095 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fbdcc1-0eec-41d8-938b-fecb6949148e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:54.987772 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fbdcc1-0eec-41d8-938b-fecb6949148e admin admin] REQUEST_METHOD: `PUT` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:54.988048 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fbdcc1-0eec-41d8-938b-fecb6949148e admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:54.988326 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fbdcc1-0eec-41d8-938b-fecb6949148e admin admin] PATH_INFO: `/v3/projects/0384852dbc38442cba25983c5abb7126/users/bc9712a3e2a3459c9147883548db8fdc/roles/62d96ac9513c452894fc67b442cfc59f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:54.990693 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fbdcc1-0eec-41d8-938b-fecb6949148e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0384852dbc38442cba25983c5abb7126, user_id=bc9712a3e2a3459c9147883548db8fdc, role_id=62d96ac9513c452894fc67b442cfc59f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:54.993305 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fbdcc1-0eec-41d8-938b-fecb6949148e admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:55.004954 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1429/2868] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:49:54 2022] PUT /identity/v3/projects/0384852dbc38442cba25983c5abb7126/users/bc9712a3e2a3459c9147883548db8fdc/roles/62d96ac9513c452894fc67b442cfc59f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:49:55.015176 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db719c2-57ef-438e-9a4d-e4dd5019b672 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:55.015451 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db719c2-57ef-438e-9a4d-e4dd5019b672 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:55.015721 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db719c2-57ef-438e-9a4d-e4dd5019b672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:55.044087 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-7db719c2-57ef-438e-9a4d-e4dd5019b672 None None] MFA Rules not processed for user `bc9712a3e2a3459c9147883548db8fdc`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:49:55.045322 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7db719c2-57ef-438e-9a4d-e4dd5019b672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:55.117916 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1440/2869] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:49:55 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:49:55.128973 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fbdcc1-0eec-41d8-938b-fecb6949148e admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:55.131131 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-77fbdcc1-0eec-41d8-938b-fecb6949148e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:55.188505 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fbdcc1-0eec-41d8-938b-fecb6949148e admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:55.192621 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0baceb4-5595-49ba-bc8d-d8f5b07124a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:55.193214 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0baceb4-5595-49ba-bc8d-d8f5b07124a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:55.193496 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0baceb4-5595-49ba-bc8d-d8f5b07124a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:55.193761 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0baceb4-5595-49ba-bc8d-d8f5b07124a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:55.248757 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0baceb4-5595-49ba-bc8d-d8f5b07124a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:55.250268 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0baceb4-5595-49ba-bc8d-d8f5b07124a9 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:55.259672 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1430/2870] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:49:55 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:49:56.654421 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db719c2-57ef-438e-9a4d-e4dd5019b672 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:56.655811 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-7db719c2-57ef-438e-9a4d-e4dd5019b672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:56.697121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:56.773625 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db719c2-57ef-438e-9a4d-e4dd5019b672 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:56.778662 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1826d244-ed89-40c0-837c-23eccb4452d6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de934d0105bd4d01ba6c06a4a0d36704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:56.778844 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0baceb4-5595-49ba-bc8d-d8f5b07124a9 service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:56.779327 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1826d244-ed89-40c0-837c-23eccb4452d6 service placement] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:56.779536 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1826d244-ed89-40c0-837c-23eccb4452d6 service placement] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:56.779794 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1826d244-ed89-40c0-837c-23eccb4452d6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:56.780361 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e0baceb4-5595-49ba-bc8d-d8f5b07124a9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:56.781443 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-1826d244-ed89-40c0-837c-23eccb4452d6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:49:56.793881 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1826d244-ed89-40c0-837c-23eccb4452d6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:56.795512 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1826d244-ed89-40c0-837c-23eccb4452d6 service placement] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:56.806471 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1441/2871] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:49:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:49:56.811219 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:49:56.821702 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0baceb4-5595-49ba-bc8d-d8f5b07124a9 service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:56.825631 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb950f0a-ea73-4df0-8a21-bb63a5b87e90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:56.826277 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb950f0a-ea73-4df0-8a21-bb63a5b87e90 service cinder] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:56.826556 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb950f0a-ea73-4df0-8a21-bb63a5b87e90 service cinder] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:56.826826 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb950f0a-ea73-4df0-8a21-bb63a5b87e90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:56.883182 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb950f0a-ea73-4df0-8a21-bb63a5b87e90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:56.884598 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb950f0a-ea73-4df0-8a21-bb63a5b87e90 service cinder] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:56.893367 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1431/2872] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:49:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:49:57.460876 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1826d244-ed89-40c0-837c-23eccb4452d6 service placement] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:49:57.470708 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1826d244-ed89-40c0-837c-23eccb4452d6 service placement] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:49:57.474270 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9915052-7941-4c0f-a45e-8c9b455ba4e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:49:57.474862 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9915052-7941-4c0f-a45e-8c9b455ba4e9 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:49:57.475141 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9915052-7941-4c0f-a45e-8c9b455ba4e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:49:57.475413 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9915052-7941-4c0f-a45e-8c9b455ba4e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:49:57.485536 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9915052-7941-4c0f-a45e-8c9b455ba4e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:49:57.487749 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9915052-7941-4c0f-a45e-8c9b455ba4e9 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:49:57.497372 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1442/2873] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:49:57 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:50:01.434384 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb950f0a-ea73-4df0-8a21-bb63a5b87e90 service cinder] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:01.503167 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb950f0a-ea73-4df0-8a21-bb63a5b87e90 service cinder] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:01.507913 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39acde8d-b80b-413a-a0bf-5269447fe619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:01.508644 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39acde8d-b80b-413a-a0bf-5269447fe619 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:01.508838 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39acde8d-b80b-413a-a0bf-5269447fe619 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:01.509112 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39acde8d-b80b-413a-a0bf-5269447fe619 admin admin] PATH_INFO: `/v3/users/6241921540904e3880e55f7b819a1959` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:01.510258 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39acde8d-b80b-413a-a0bf-5269447fe619 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6241921540904e3880e55f7b819a1959)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:01.512325 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39acde8d-b80b-413a-a0bf-5269447fe619 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:01.562111 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-39acde8d-b80b-413a-a0bf-5269447fe619 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6241921540904e3880e55f7b819a1959'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:01.569526 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-39acde8d-b80b-413a-a0bf-5269447fe619 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6241921540904e3880e55f7b819a1959'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:01.570211 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-39acde8d-b80b-413a-a0bf-5269447fe619 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6241921540904e3880e55f7b819a1959'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:01.577571 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-39acde8d-b80b-413a-a0bf-5269447fe619 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6241921540904e3880e55f7b819a1959'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:01.578235 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-39acde8d-b80b-413a-a0bf-5269447fe619 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6241921540904e3880e55f7b819a1959'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:01.595322 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1432/2874] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:50:01 2022] DELETE /identity/v3/users/6241921540904e3880e55f7b819a1959 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:50:01.599941 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a401041-738d-430a-bdc4-22afa6da8a77 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:01.600419 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a401041-738d-430a-bdc4-22afa6da8a77 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:01.600616 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a401041-738d-430a-bdc4-22afa6da8a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:01.619950 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-0a401041-738d-430a-bdc4-22afa6da8a77 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:50:01.621126 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0a401041-738d-430a-bdc4-22afa6da8a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:01.680395 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1443/2875] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:50:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:50:01.690780 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39acde8d-b80b-413a-a0bf-5269447fe619 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:01.692211 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-39acde8d-b80b-413a-a0bf-5269447fe619 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:01.713716 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:50:01.755134 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39acde8d-b80b-413a-a0bf-5269447fe619 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:01.759282 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c0c338-9d07-43a6-8d9f-a49bde7f7bcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:01.759897 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c0c338-9d07-43a6-8d9f-a49bde7f7bcd service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:01.760174 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c0c338-9d07-43a6-8d9f-a49bde7f7bcd service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:01.760510 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c0c338-9d07-43a6-8d9f-a49bde7f7bcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:01.771438 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c0c338-9d07-43a6-8d9f-a49bde7f7bcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:01.772887 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c0c338-9d07-43a6-8d9f-a49bde7f7bcd service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:01.783018 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1433/2876] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:50:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:50:02.051930 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a401041-738d-430a-bdc4-22afa6da8a77 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:02.053350 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-0a401041-738d-430a-bdc4-22afa6da8a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:02.065961 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a401041-738d-430a-bdc4-22afa6da8a77 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:02.070545 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ffef7-636a-4110-b9c7-0d29bb56e1f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:02.071175 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ffef7-636a-4110-b9c7-0d29bb56e1f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:02.071457 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ffef7-636a-4110-b9c7-0d29bb56e1f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:02.071731 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ffef7-636a-4110-b9c7-0d29bb56e1f2 admin admin] PATH_INFO: `/v3/projects/a0260d8fef7046bb9dac0d4aadc5e401` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:02.072886 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196ffef7-636a-4110-b9c7-0d29bb56e1f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0260d8fef7046bb9dac0d4aadc5e401)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:02.074883 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196ffef7-636a-4110-b9c7-0d29bb56e1f2 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:02.125206 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-196ffef7-636a-4110-b9c7-0d29bb56e1f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0260d8fef7046bb9dac0d4aadc5e401', 'request_id': 'req-196ffef7-636a-4110-b9c7-0d29bb56e1f2'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:02.131965 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-196ffef7-636a-4110-b9c7-0d29bb56e1f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0260d8fef7046bb9dac0d4aadc5e401', 'request_id': 'req-196ffef7-636a-4110-b9c7-0d29bb56e1f2'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:02.139591 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-196ffef7-636a-4110-b9c7-0d29bb56e1f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0260d8fef7046bb9dac0d4aadc5e401', 'request_id': 'req-196ffef7-636a-4110-b9c7-0d29bb56e1f2'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:02.150369 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-196ffef7-636a-4110-b9c7-0d29bb56e1f2 admin admin] The token cache is being invalidate because project a0260d8fef7046bb9dac0d4aadc5e401 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:50:02.150654 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-196ffef7-636a-4110-b9c7-0d29bb56e1f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:02.151271 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-196ffef7-636a-4110-b9c7-0d29bb56e1f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:02.152730 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1444/2877] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:50:02 2022] DELETE /identity/v3/projects/a0260d8fef7046bb9dac0d4aadc5e401 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:50:02.156767 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c0c338-9d07-43a6-8d9f-a49bde7f7bcd service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:02.166253 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c0c338-9d07-43a6-8d9f-a49bde7f7bcd service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:02.221297 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29424375-8064-4da0-b7f6-55b91ec8ff0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:02.221839 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29424375-8064-4da0-b7f6-55b91ec8ff0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:02.222204 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29424375-8064-4da0-b7f6-55b91ec8ff0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:02.222474 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29424375-8064-4da0-b7f6-55b91ec8ff0c admin admin] PATH_INFO: `/v3/users/29f2486a0a484da7a623351baf37876b` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:02.223563 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29424375-8064-4da0-b7f6-55b91ec8ff0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=29f2486a0a484da7a623351baf37876b)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:02.225454 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29424375-8064-4da0-b7f6-55b91ec8ff0c admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:02.275429 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-29424375-8064-4da0-b7f6-55b91ec8ff0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29f2486a0a484da7a623351baf37876b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:02.282456 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-29424375-8064-4da0-b7f6-55b91ec8ff0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29f2486a0a484da7a623351baf37876b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:02.283133 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-29424375-8064-4da0-b7f6-55b91ec8ff0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29f2486a0a484da7a623351baf37876b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:02.291388 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-29424375-8064-4da0-b7f6-55b91ec8ff0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29f2486a0a484da7a623351baf37876b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:02.292091 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-29424375-8064-4da0-b7f6-55b91ec8ff0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29f2486a0a484da7a623351baf37876b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:02.313082 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1434/2878] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:50:02 2022] DELETE /identity/v3/users/29f2486a0a484da7a623351baf37876b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:50:02.586051 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ffef7-636a-4110-b9c7-0d29bb56e1f2 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:02.587603 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-196ffef7-636a-4110-b9c7-0d29bb56e1f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:02.601262 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ffef7-636a-4110-b9c7-0d29bb56e1f2 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:02.605411 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:50:02.661445 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2fc2bb-2563-47fa-8b90-a80b4aabc4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:02.662139 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2fc2bb-2563-47fa-8b90-a80b4aabc4d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:02.662419 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2fc2bb-2563-47fa-8b90-a80b4aabc4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:02.662687 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2fc2bb-2563-47fa-8b90-a80b4aabc4d8 admin admin] PATH_INFO: `/v3/projects/9f138b87ff3e4ac29678ef3a5ce353c8` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:02.663772 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2fc2bb-2563-47fa-8b90-a80b4aabc4d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f138b87ff3e4ac29678ef3a5ce353c8)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:02.665682 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2fc2bb-2563-47fa-8b90-a80b4aabc4d8 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:02.710637 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ac2fc2bb-2563-47fa-8b90-a80b4aabc4d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f138b87ff3e4ac29678ef3a5ce353c8', 'request_id': 'req-ac2fc2bb-2563-47fa-8b90-a80b4aabc4d8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:02.716579 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ac2fc2bb-2563-47fa-8b90-a80b4aabc4d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f138b87ff3e4ac29678ef3a5ce353c8', 'request_id': 'req-ac2fc2bb-2563-47fa-8b90-a80b4aabc4d8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:02.725208 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ac2fc2bb-2563-47fa-8b90-a80b4aabc4d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f138b87ff3e4ac29678ef3a5ce353c8', 'request_id': 'req-ac2fc2bb-2563-47fa-8b90-a80b4aabc4d8'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:02.737069 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ac2fc2bb-2563-47fa-8b90-a80b4aabc4d8 admin admin] The token cache is being invalidate because project 9f138b87ff3e4ac29678ef3a5ce353c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:50:02.737408 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ac2fc2bb-2563-47fa-8b90-a80b4aabc4d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:02.738064 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-ac2fc2bb-2563-47fa-8b90-a80b4aabc4d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:02.739527 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1445/2879] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:50:02 2022] DELETE /identity/v3/projects/9f138b87ff3e4ac29678ef3a5ce353c8 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:50:02.743505 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29424375-8064-4da0-b7f6-55b91ec8ff0c admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:02.744878 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-29424375-8064-4da0-b7f6-55b91ec8ff0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:02.757452 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29424375-8064-4da0-b7f6-55b91ec8ff0c admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:02.796145 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:50:02.815786 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666aaec7-5158-422e-8a6d-6e254c0f7040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:02.816508 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666aaec7-5158-422e-8a6d-6e254c0f7040 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:02.816829 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666aaec7-5158-422e-8a6d-6e254c0f7040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:02.817097 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666aaec7-5158-422e-8a6d-6e254c0f7040 admin admin] PATH_INFO: `/v3/users/bc9712a3e2a3459c9147883548db8fdc` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:02.818358 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666aaec7-5158-422e-8a6d-6e254c0f7040 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc9712a3e2a3459c9147883548db8fdc)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:02.820294 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666aaec7-5158-422e-8a6d-6e254c0f7040 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:02.864857 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-666aaec7-5158-422e-8a6d-6e254c0f7040 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc9712a3e2a3459c9147883548db8fdc'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:02.871521 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-666aaec7-5158-422e-8a6d-6e254c0f7040 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc9712a3e2a3459c9147883548db8fdc'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:02.872167 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-666aaec7-5158-422e-8a6d-6e254c0f7040 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc9712a3e2a3459c9147883548db8fdc'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:02.879570 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-666aaec7-5158-422e-8a6d-6e254c0f7040 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc9712a3e2a3459c9147883548db8fdc'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:02.880201 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-666aaec7-5158-422e-8a6d-6e254c0f7040 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc9712a3e2a3459c9147883548db8fdc'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:02.897426 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1435/2880] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:50:02 2022] DELETE /identity/v3/users/bc9712a3e2a3459c9147883548db8fdc => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:50:03.170463 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2fc2bb-2563-47fa-8b90-a80b4aabc4d8 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:03.172028 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ac2fc2bb-2563-47fa-8b90-a80b4aabc4d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:03.186730 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2fc2bb-2563-47fa-8b90-a80b4aabc4d8 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:03.244927 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8f6542-fd03-46a6-ba60-b5c487d0c082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:03.245464 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f6542-fd03-46a6-ba60-b5c487d0c082 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:03.245744 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f6542-fd03-46a6-ba60-b5c487d0c082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:03.246119 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f6542-fd03-46a6-ba60-b5c487d0c082 admin admin] PATH_INFO: `/v3/projects/0384852dbc38442cba25983c5abb7126` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:03.247227 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8f6542-fd03-46a6-ba60-b5c487d0c082 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0384852dbc38442cba25983c5abb7126)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:03.249148 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8f6542-fd03-46a6-ba60-b5c487d0c082 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:03.302575 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:50:03.314015 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-0d8f6542-fd03-46a6-ba60-b5c487d0c082 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0384852dbc38442cba25983c5abb7126', 'request_id': 'req-0d8f6542-fd03-46a6-ba60-b5c487d0c082'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:03.321483 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-0d8f6542-fd03-46a6-ba60-b5c487d0c082 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0384852dbc38442cba25983c5abb7126', 'request_id': 'req-0d8f6542-fd03-46a6-ba60-b5c487d0c082'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:03.332249 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-0d8f6542-fd03-46a6-ba60-b5c487d0c082 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0384852dbc38442cba25983c5abb7126', 'request_id': 'req-0d8f6542-fd03-46a6-ba60-b5c487d0c082'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:03.343553 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-0d8f6542-fd03-46a6-ba60-b5c487d0c082 admin admin] The token cache is being invalidate because project 0384852dbc38442cba25983c5abb7126 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:50:03.343896 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-0d8f6542-fd03-46a6-ba60-b5c487d0c082 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:03.344513 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-0d8f6542-fd03-46a6-ba60-b5c487d0c082 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:03.346068 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1446/2881] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:50:03 2022] DELETE /identity/v3/projects/0384852dbc38442cba25983c5abb7126 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:50:03.352753 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a4b6f5-881b-4f53-aa98-ea56e3f229f7 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:03.353142 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a4b6f5-881b-4f53-aa98-ea56e3f229f7 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:03.353415 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a4b6f5-881b-4f53-aa98-ea56e3f229f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:03.373843 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-50a4b6f5-881b-4f53-aa98-ea56e3f229f7 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:50:03.375182 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-50a4b6f5-881b-4f53-aa98-ea56e3f229f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:03.433365 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1436/2882] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:50:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:50:03.442998 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c48532-4cc9-4df6-9543-4e9365455a04 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:03.443308 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c48532-4cc9-4df6-9543-4e9365455a04 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:03.443580 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c48532-4cc9-4df6-9543-4e9365455a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:03.464008 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-e1c48532-4cc9-4df6-9543-4e9365455a04 None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:50:03.465225 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-e1c48532-4cc9-4df6-9543-4e9365455a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:03.473075 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1447/2883] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:50:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:50:03.477409 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a4b6f5-881b-4f53-aa98-ea56e3f229f7 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:03.487407 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a4b6f5-881b-4f53-aa98-ea56e3f229f7 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:03.491917 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bb5ffe-f640-4481-9f5b-d279839e2221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:03.492518 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb5ffe-f640-4481-9f5b-d279839e2221 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:03.492790 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb5ffe-f640-4481-9f5b-d279839e2221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:03.493056 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb5ffe-f640-4481-9f5b-d279839e2221 admin admin] PATH_INFO: `/v3/domains` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:03.493689 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bb5ffe-f640-4481-9f5b-d279839e2221 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81604) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 28 15:50:03.494018 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bb5ffe-f640-4481-9f5b-d279839e2221 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:03.495686 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bb5ffe-f640-4481-9f5b-d279839e2221 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:03.507753 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1437/2884] 10.222.0.52 () {58 vars in 1153 bytes} [Wed Sep 28 15:50:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 28 15:50:03.512138 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c48532-4cc9-4df6-9543-4e9365455a04 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:03.522751 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c48532-4cc9-4df6-9543-4e9365455a04 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:03.527456 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ad4ad6-f436-4808-822c-dd7106d73d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:03.528061 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad4ad6-f436-4808-822c-dd7106d73d04 admin admin] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:03.528338 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad4ad6-f436-4808-822c-dd7106d73d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:03.528610 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad4ad6-f436-4808-822c-dd7106d73d04 admin admin] PATH_INFO: `/v3/projects` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:03.529254 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ad4ad6-f436-4808-822c-dd7106d73d04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:03.531247 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ad4ad6-f436-4808-822c-dd7106d73d04 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:03.568953 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1448/2885] 10.222.0.52 () {60 vars in 1153 bytes} [Wed Sep 28 15:50:03 2022] POST /identity/v3/projects => generated 375 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 28 15:50:03.574555 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bb5ffe-f640-4481-9f5b-d279839e2221 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:03.584839 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bb5ffe-f640-4481-9f5b-d279839e2221 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:03.630669 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:50:03.644935 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f80cf-f177-418b-9ee6-afebcaa92dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:03.645570 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f80cf-f177-418b-9ee6-afebcaa92dd4 admin admin] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:03.645870 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f80cf-f177-418b-9ee6-afebcaa92dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:03.646146 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f80cf-f177-418b-9ee6-afebcaa92dd4 admin admin] PATH_INFO: `/v3/users` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:03.646771 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69f80cf-f177-418b-9ee6-afebcaa92dd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:03.648697 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69f80cf-f177-418b-9ee6-afebcaa92dd4 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:03.669071 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1438/2886] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Sep 28 15:50:03 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 28 15:50:03.673669 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ad4ad6-f436-4808-822c-dd7106d73d04 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:03.683808 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ad4ad6-f436-4808-822c-dd7106d73d04 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:03.688871 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf565f-d451-46fb-bc7e-2ab6692414dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:03.689461 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf565f-d451-46fb-bc7e-2ab6692414dc admin admin] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:03.689737 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf565f-d451-46fb-bc7e-2ab6692414dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:03.690031 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf565f-d451-46fb-bc7e-2ab6692414dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:03.690655 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf565f-d451-46fb-bc7e-2ab6692414dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:03.692241 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf565f-d451-46fb-bc7e-2ab6692414dc admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:03.701863 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1449/2887] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 28 15:50:03.705592 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f80cf-f177-418b-9ee6-afebcaa92dd4 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:03.715476 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f80cf-f177-418b-9ee6-afebcaa92dd4 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:03.720122 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289fe833-a3a5-4814-8488-a0e835ce8829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:03.720716 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289fe833-a3a5-4814-8488-a0e835ce8829 admin admin] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:03.720992 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289fe833-a3a5-4814-8488-a0e835ce8829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:03.721260 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289fe833-a3a5-4814-8488-a0e835ce8829 admin admin] PATH_INFO: `/v3/roles` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:03.721918 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289fe833-a3a5-4814-8488-a0e835ce8829 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:03.723591 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289fe833-a3a5-4814-8488-a0e835ce8829 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:03.733695 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1439/2888] 10.222.0.52 () {58 vars in 1122 bytes} [Wed Sep 28 15:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 28 15:50:03.738918 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf565f-d451-46fb-bc7e-2ab6692414dc admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:03.749016 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf565f-d451-46fb-bc7e-2ab6692414dc admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:03.754821 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea4bcf9-20d9-404b-84b5-fdb164f2770f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:03.755688 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4bcf9-20d9-404b-84b5-fdb164f2770f admin admin] REQUEST_METHOD: `PUT` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:03.755973 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4bcf9-20d9-404b-84b5-fdb164f2770f admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:03.756252 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4bcf9-20d9-404b-84b5-fdb164f2770f admin admin] PATH_INFO: `/v3/projects/bf6381246ada4ba084f7cc0d514def4f/users/e58268726b07457884cd7e250437ec78/roles/e152a36b5a1b470b912b1d705462f512` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:03.771757 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea4bcf9-20d9-404b-84b5-fdb164f2770f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf6381246ada4ba084f7cc0d514def4f, user_id=e58268726b07457884cd7e250437ec78, role_id=e152a36b5a1b470b912b1d705462f512)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:03.774432 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea4bcf9-20d9-404b-84b5-fdb164f2770f admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:03.787099 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1450/2889] 10.222.0.52 () {60 vars in 1483 bytes} [Wed Sep 28 15:50:03 2022] PUT /identity/v3/projects/bf6381246ada4ba084f7cc0d514def4f/users/e58268726b07457884cd7e250437ec78/roles/e152a36b5a1b470b912b1d705462f512 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:50:03.797801 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd657c-18ad-4860-9887-c2c5b8f72567 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:03.798297 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd657c-18ad-4860-9887-c2c5b8f72567 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:03.798442 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd657c-18ad-4860-9887-c2c5b8f72567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:03.826796 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-ebbd657c-18ad-4860-9887-c2c5b8f72567 None None] MFA Rules not processed for user `e58268726b07457884cd7e250437ec78`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:50:03.827999 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ebbd657c-18ad-4860-9887-c2c5b8f72567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:03.891772 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1440/2890] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:50:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:50:03.902305 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea4bcf9-20d9-404b-84b5-fdb164f2770f admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:03.903963 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-dea4bcf9-20d9-404b-84b5-fdb164f2770f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:03.960061 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea4bcf9-20d9-404b-84b5-fdb164f2770f admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:03.963715 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81049c37-28e8-48ca-9f93-26e8f0acbda5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01833709177e4e68a2fa52fb9963d3bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:03.964311 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81049c37-28e8-48ca-9f93-26e8f0acbda5 service cinder] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:03.964592 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81049c37-28e8-48ca-9f93-26e8f0acbda5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:03.964866 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81049c37-28e8-48ca-9f93-26e8f0acbda5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:03.973435 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81049c37-28e8-48ca-9f93-26e8f0acbda5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:03.974904 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81049c37-28e8-48ca-9f93-26e8f0acbda5 service cinder] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:03.983698 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1451/2891] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:50:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:50:05.337874 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25eb0bf-fb75-48dc-94ae-9d76586eb21f None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:05.338388 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25eb0bf-fb75-48dc-94ae-9d76586eb21f None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:05.338519 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25eb0bf-fb75-48dc-94ae-9d76586eb21f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:05.358525 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-c25eb0bf-fb75-48dc-94ae-9d76586eb21f None None] MFA Rules not processed for user `e58268726b07457884cd7e250437ec78`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:50:05.360191 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-c25eb0bf-fb75-48dc-94ae-9d76586eb21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:05.367145 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1441/2892] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:50:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:50:09.293461 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81049c37-28e8-48ca-9f93-26e8f0acbda5 service cinder] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:09.295130 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-81049c37-28e8-48ca-9f93-26e8f0acbda5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:09.353706 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81049c37-28e8-48ca-9f93-26e8f0acbda5 service cinder] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:09.357324 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37fc90d-86da-4268-b65a-64393ca5b297 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:09.357947 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37fc90d-86da-4268-b65a-64393ca5b297 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:09.358228 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37fc90d-86da-4268-b65a-64393ca5b297 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:09.358499 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37fc90d-86da-4268-b65a-64393ca5b297 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:09.361321 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d37fc90d-86da-4268-b65a-64393ca5b297 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:09.381955 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:50:09.455277 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37fc90d-86da-4268-b65a-64393ca5b297 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:09.457344 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37fc90d-86da-4268-b65a-64393ca5b297 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:09.470816 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1452/2893] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:50:09 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:50:13.262333 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25eb0bf-fb75-48dc-94ae-9d76586eb21f None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:13.275281 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25eb0bf-fb75-48dc-94ae-9d76586eb21f None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:13.278970 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1915b41f-9004-49ed-800e-285d2717e325 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:13.279574 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915b41f-9004-49ed-800e-285d2717e325 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:13.279852 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915b41f-9004-49ed-800e-285d2717e325 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:13.280121 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915b41f-9004-49ed-800e-285d2717e325 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:13.282668 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-1915b41f-9004-49ed-800e-285d2717e325 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:13.353605 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1915b41f-9004-49ed-800e-285d2717e325 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:13.355119 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1915b41f-9004-49ed-800e-285d2717e325 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:13.366594 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1442/2894] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:50:13 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:50:19.465602 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=20 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:50:22.274850 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e19d4-e751-4600-9e60-1b774d0cd4ec None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:22.275431 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e19d4-e751-4600-9e60-1b774d0cd4ec None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:22.275431 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e19d4-e751-4600-9e60-1b774d0cd4ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:22.294676 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-092e19d4-e751-4600-9e60-1b774d0cd4ec None None] MFA Rules not processed for user `c81e5fb007614cf2992f5e02e85d6307`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:50:22.295873 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-092e19d4-e751-4600-9e60-1b774d0cd4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:22.350349 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1453/2895] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Sep 28 15:50:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:50:22.372348 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1915b41f-9004-49ed-800e-285d2717e325 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:22.373811 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-1915b41f-9004-49ed-800e-285d2717e325 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:22.419108 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:50:22.444023 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1915b41f-9004-49ed-800e-285d2717e325 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:22.452131 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ab611b-314c-4644-bde4-870bbeb85964 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '328051fe7607464f8f5dc0dcc01a817a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:22.453195 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ab611b-314c-4644-bde4-870bbeb85964 service swift] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:22.453702 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ab611b-314c-4644-bde4-870bbeb85964 service swift] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:22.454150 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ab611b-314c-4644-bde4-870bbeb85964 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:22.462899 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ab611b-314c-4644-bde4-870bbeb85964 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:22.464460 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ab611b-314c-4644-bde4-870bbeb85964 service swift] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:22.472782 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1443/2896] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 15:50:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 15:50:22.975153 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca9ff1-e10a-4864-bca9-0251e0751c34 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:22.975442 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca9ff1-e10a-4864-bca9-0251e0751c34 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:22.975714 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca9ff1-e10a-4864-bca9-0251e0751c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:22.995396 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-b3ca9ff1-e10a-4864-bca9-0251e0751c34 None None] MFA Rules not processed for user `c81e5fb007614cf2992f5e02e85d6307`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:50:22.996580 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-b3ca9ff1-e10a-4864-bca9-0251e0751c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:23.003610 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1454/2897] 10.222.0.52 () {58 vars in 967 bytes} [Wed Sep 28 15:50:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:50:23.015888 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ab611b-314c-4644-bde4-870bbeb85964 service swift] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:23.026148 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ab611b-314c-4644-bde4-870bbeb85964 service swift] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:23.031749 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cf2e03-541b-46c3-b170-1a8f394362e9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '328051fe7607464f8f5dc0dcc01a817a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:23.032345 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf2e03-541b-46c3-b170-1a8f394362e9 service swift] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:23.032617 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf2e03-541b-46c3-b170-1a8f394362e9 service swift] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:23.032883 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf2e03-541b-46c3-b170-1a8f394362e9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:23.040981 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cf2e03-541b-46c3-b170-1a8f394362e9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:23.042585 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cf2e03-541b-46c3-b170-1a8f394362e9 service swift] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:23.050471 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1444/2898] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 15:50:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 15:50:23.270470 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfccd238-02c2-4569-8aaa-87cbf653fb53 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:23.270727 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfccd238-02c2-4569-8aaa-87cbf653fb53 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:23.271003 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfccd238-02c2-4569-8aaa-87cbf653fb53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:23.291098 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-cfccd238-02c2-4569-8aaa-87cbf653fb53 None None] MFA Rules not processed for user `c81e5fb007614cf2992f5e02e85d6307`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:50:23.292318 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-cfccd238-02c2-4569-8aaa-87cbf653fb53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:23.299734 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1455/2899] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Sep 28 15:50:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:50:23.311428 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cf2e03-541b-46c3-b170-1a8f394362e9 service swift] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:23.321912 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cf2e03-541b-46c3-b170-1a8f394362e9 service swift] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:23.326926 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4c877-eb88-41f2-9005-58364483d10e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '328051fe7607464f8f5dc0dcc01a817a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:23.327995 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4c877-eb88-41f2-9005-58364483d10e service swift] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:23.328267 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4c877-eb88-41f2-9005-58364483d10e service swift] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:23.328604 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4c877-eb88-41f2-9005-58364483d10e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:23.336581 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c4c877-eb88-41f2-9005-58364483d10e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:23.338156 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c4c877-eb88-41f2-9005-58364483d10e service swift] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:23.346612 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1445/2900] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 15:50:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 15:50:23.662227 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c64db5-aceb-43f4-ad55-6b2f7d9fd93f None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:23.662509 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c64db5-aceb-43f4-ad55-6b2f7d9fd93f None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:23.662781 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c64db5-aceb-43f4-ad55-6b2f7d9fd93f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:23.681832 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-d7c64db5-aceb-43f4-ad55-6b2f7d9fd93f None None] MFA Rules not processed for user `c81e5fb007614cf2992f5e02e85d6307`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:50:23.683012 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d7c64db5-aceb-43f4-ad55-6b2f7d9fd93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:23.690250 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1456/2901] 10.222.0.52 () {58 vars in 967 bytes} [Wed Sep 28 15:50:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:50:23.702186 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4c877-eb88-41f2-9005-58364483d10e service swift] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:23.713224 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4c877-eb88-41f2-9005-58364483d10e service swift] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:23.718978 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2016bb8d-7d7f-47fe-be49-4ec15fecdb3e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '328051fe7607464f8f5dc0dcc01a817a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:23.719589 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2016bb8d-7d7f-47fe-be49-4ec15fecdb3e service swift] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:23.719870 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2016bb8d-7d7f-47fe-be49-4ec15fecdb3e service swift] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:23.720147 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2016bb8d-7d7f-47fe-be49-4ec15fecdb3e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:23.728007 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2016bb8d-7d7f-47fe-be49-4ec15fecdb3e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:23.729562 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2016bb8d-7d7f-47fe-be49-4ec15fecdb3e service swift] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:23.738732 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1446/2902] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 15:50:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 28 15:50:25.976818 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c64db5-aceb-43f4-ad55-6b2f7d9fd93f None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:26.038321 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c64db5-aceb-43f4-ad55-6b2f7d9fd93f None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:26.042738 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4f140-d123-4c6d-85e1-3c4114f29499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:26.043354 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4f140-d123-4c6d-85e1-3c4114f29499 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:26.043633 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4f140-d123-4c6d-85e1-3c4114f29499 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:26.043907 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4f140-d123-4c6d-85e1-3c4114f29499 admin admin] PATH_INFO: `/v3/users/e58268726b07457884cd7e250437ec78` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:26.045009 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac4f140-d123-4c6d-85e1-3c4114f29499 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e58268726b07457884cd7e250437ec78)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:26.047460 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac4f140-d123-4c6d-85e1-3c4114f29499 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:26.094073 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2ac4f140-d123-4c6d-85e1-3c4114f29499 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e58268726b07457884cd7e250437ec78'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:26.094721 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2ac4f140-d123-4c6d-85e1-3c4114f29499 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e58268726b07457884cd7e250437ec78'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:26.106738 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2ac4f140-d123-4c6d-85e1-3c4114f29499 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e58268726b07457884cd7e250437ec78'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:26.114015 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2ac4f140-d123-4c6d-85e1-3c4114f29499 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e58268726b07457884cd7e250437ec78'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:26.120835 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-2ac4f140-d123-4c6d-85e1-3c4114f29499 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e58268726b07457884cd7e250437ec78'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:26.127183 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1457/2903] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:50:25 2022] DELETE /identity/v3/users/e58268726b07457884cd7e250437ec78 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:50:26.131815 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bea1ce-d228-4baf-bb42-857d0d65eb7c None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:26.132323 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bea1ce-d228-4baf-bb42-857d0d65eb7c None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:26.132441 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bea1ce-d228-4baf-bb42-857d0d65eb7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:26.150982 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-76bea1ce-d228-4baf-bb42-857d0d65eb7c None None] MFA Rules not processed for user `0fdbcf7ba1534603b18de5dbaf96574f`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:50:26.152156 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-76bea1ce-d228-4baf-bb42-857d0d65eb7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:26.213066 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1447/2904] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:50:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:50:26.222621 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4f140-d123-4c6d-85e1-3c4114f29499 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:26.224020 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2ac4f140-d123-4c6d-85e1-3c4114f29499 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:26.230506 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:50:26.283297 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4f140-d123-4c6d-85e1-3c4114f29499 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:26.286950 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d470a20e-ed0f-43d0-9dfa-c6b0264c5b66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:26.287664 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470a20e-ed0f-43d0-9dfa-c6b0264c5b66 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:26.287944 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470a20e-ed0f-43d0-9dfa-c6b0264c5b66 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:26.288217 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d470a20e-ed0f-43d0-9dfa-c6b0264c5b66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:26.298121 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d470a20e-ed0f-43d0-9dfa-c6b0264c5b66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:26.299266 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d470a20e-ed0f-43d0-9dfa-c6b0264c5b66 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:26.308555 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1458/2905] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:50:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:50:26.471888 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bea1ce-d228-4baf-bb42-857d0d65eb7c None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:50:26.473745 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-76bea1ce-d228-4baf-bb42-857d0d65eb7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:50:26.490878 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bea1ce-d228-4baf-bb42-857d0d65eb7c None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:50:26.496268 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee18d08c-ebdf-4e5d-843c-e5a94a06ecac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:50:26.497038 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee18d08c-ebdf-4e5d-843c-e5a94a06ecac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:50:26.497357 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee18d08c-ebdf-4e5d-843c-e5a94a06ecac admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:50:26.497691 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee18d08c-ebdf-4e5d-843c-e5a94a06ecac admin admin] PATH_INFO: `/v3/projects/bf6381246ada4ba084f7cc0d514def4f` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:50:26.498946 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee18d08c-ebdf-4e5d-843c-e5a94a06ecac admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf6381246ada4ba084f7cc0d514def4f)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:50:26.501138 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee18d08c-ebdf-4e5d-843c-e5a94a06ecac admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:50:26.551577 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ee18d08c-ebdf-4e5d-843c-e5a94a06ecac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf6381246ada4ba084f7cc0d514def4f', 'request_id': 'req-ee18d08c-ebdf-4e5d-843c-e5a94a06ecac'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:26.559042 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ee18d08c-ebdf-4e5d-843c-e5a94a06ecac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf6381246ada4ba084f7cc0d514def4f', 'request_id': 'req-ee18d08c-ebdf-4e5d-843c-e5a94a06ecac'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:26.565196 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ee18d08c-ebdf-4e5d-843c-e5a94a06ecac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf6381246ada4ba084f7cc0d514def4f', 'request_id': 'req-ee18d08c-ebdf-4e5d-843c-e5a94a06ecac'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:26.575291 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ee18d08c-ebdf-4e5d-843c-e5a94a06ecac admin admin] The token cache is being invalidate because project bf6381246ada4ba084f7cc0d514def4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:50:26.575571 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ee18d08c-ebdf-4e5d-843c-e5a94a06ecac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:26.576178 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-ee18d08c-ebdf-4e5d-843c-e5a94a06ecac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:50:26.577661 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1448/2906] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:50:26 2022] DELETE /identity/v3/projects/bf6381246ada4ba084f7cc0d514def4f => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:50:36.304372 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=24 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:50:36.572418 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=85,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:51:28.131553 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d470a20e-ed0f-43d0-9dfa-c6b0264c5b66 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:51:28.134506 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-d470a20e-ed0f-43d0-9dfa-c6b0264c5b66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:51:28.151744 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d470a20e-ed0f-43d0-9dfa-c6b0264c5b66 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:51:28.377046 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a19bd8c-94e6-4ab0-b994-b8a7e36d89c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:51:28.377687 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a19bd8c-94e6-4ab0-b994-b8a7e36d89c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:51:28.378007 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a19bd8c-94e6-4ab0-b994-b8a7e36d89c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:51:28.378282 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a19bd8c-94e6-4ab0-b994-b8a7e36d89c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:51:28.381390 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-2a19bd8c-94e6-4ab0-b994-b8a7e36d89c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:51:28.522963 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a19bd8c-94e6-4ab0-b994-b8a7e36d89c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:51:28.524892 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a19bd8c-94e6-4ab0-b994-b8a7e36d89c6 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:51:28.540110 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1459/2907] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:51:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:51:30.081575 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee18d08c-ebdf-4e5d-843c-e5a94a06ecac admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:51:30.084121 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ee18d08c-ebdf-4e5d-843c-e5a94a06ecac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:51:30.106195 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee18d08c-ebdf-4e5d-843c-e5a94a06ecac admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:51:30.113140 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b1114-26bf-42e4-a3ec-316d9b21da27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:51:30.115026 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b1114-26bf-42e4-a3ec-316d9b21da27 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:51:30.115510 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b1114-26bf-42e4-a3ec-316d9b21da27 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:51:30.116173 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b1114-26bf-42e4-a3ec-316d9b21da27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:51:30.119141 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-479b1114-26bf-42e4-a3ec-316d9b21da27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:51:30.158901 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479b1114-26bf-42e4-a3ec-316d9b21da27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:51:30.161454 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479b1114-26bf-42e4-a3ec-316d9b21da27 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:51:30.182892 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1449/2908] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:51:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:51:34.561984 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a19bd8c-94e6-4ab0-b994-b8a7e36d89c6 service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:51:34.571652 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a19bd8c-94e6-4ab0-b994-b8a7e36d89c6 service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:51:34.576246 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660bc6a1-b78c-430a-8719-0fcf09a68b25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:51:34.576988 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660bc6a1-b78c-430a-8719-0fcf09a68b25 service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:51:34.577382 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660bc6a1-b78c-430a-8719-0fcf09a68b25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:51:34.577755 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660bc6a1-b78c-430a-8719-0fcf09a68b25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:51:34.579932 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-660bc6a1-b78c-430a-8719-0fcf09a68b25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:51:34.592008 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660bc6a1-b78c-430a-8719-0fcf09a68b25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:51:34.593744 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660bc6a1-b78c-430a-8719-0fcf09a68b25 service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:51:34.605976 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1460/2909] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:51:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:51:35.188614 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b1114-26bf-42e4-a3ec-316d9b21da27 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:51:35.191050 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-479b1114-26bf-42e4-a3ec-316d9b21da27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:51:35.203995 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b1114-26bf-42e4-a3ec-316d9b21da27 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:51:35.207185 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5056a71-8369-48bf-91db-1a503fbc54d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:51:35.207799 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5056a71-8369-48bf-91db-1a503fbc54d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:51:35.208083 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5056a71-8369-48bf-91db-1a503fbc54d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:51:35.208360 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5056a71-8369-48bf-91db-1a503fbc54d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:51:35.217035 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-f5056a71-8369-48bf-91db-1a503fbc54d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:51:35.243737 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5056a71-8369-48bf-91db-1a503fbc54d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:51:35.245740 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5056a71-8369-48bf-91db-1a503fbc54d5 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:51:35.263018 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1450/2910] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:51:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 28 15:51:38.898241 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f19d823-ffe1-45bc-b98d-5a9229eea2e4 None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:51:38.898773 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f19d823-ffe1-45bc-b98d-5a9229eea2e4 None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:51:38.898773 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f19d823-ffe1-45bc-b98d-5a9229eea2e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:51:38.920212 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-6f19d823-ffe1-45bc-b98d-5a9229eea2e4 None None] MFA Rules not processed for user `c81e5fb007614cf2992f5e02e85d6307`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:51:38.921464 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6f19d823-ffe1-45bc-b98d-5a9229eea2e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:51:38.979349 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1461/2911] 10.222.0.52 () {58 vars in 967 bytes} [Wed Sep 28 15:51:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 28 15:51:39.033824 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5056a71-8369-48bf-91db-1a503fbc54d5 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:51:39.035484 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-f5056a71-8369-48bf-91db-1a503fbc54d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:51:39.145540 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5056a71-8369-48bf-91db-1a503fbc54d5 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:51:39.151503 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81c2593-02e3-410d-997b-ef04adf48034 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '328051fe7607464f8f5dc0dcc01a817a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:51:39.152130 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c2593-02e3-410d-997b-ef04adf48034 service swift] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:51:39.152562 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c2593-02e3-410d-997b-ef04adf48034 service swift] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:51:39.152842 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c2593-02e3-410d-997b-ef04adf48034 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:51:39.161680 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81c2593-02e3-410d-997b-ef04adf48034 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:51:39.163461 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81c2593-02e3-410d-997b-ef04adf48034 service swift] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:51:39.174153 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1451/2912] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Sep 28 15:51:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 28 15:51:45.917326 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f19d823-ffe1-45bc-b98d-5a9229eea2e4 None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:51:45.918822 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6f19d823-ffe1-45bc-b98d-5a9229eea2e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:51:45.961315 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=100 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:51:45.995692 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f19d823-ffe1-45bc-b98d-5a9229eea2e4 None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:51:46.000681 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2645d4b-d6e6-4904-9a24-1ea2eb418d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:51:46.001581 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2645d4b-d6e6-4904-9a24-1ea2eb418d9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:51:46.001900 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2645d4b-d6e6-4904-9a24-1ea2eb418d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:51:46.002172 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2645d4b-d6e6-4904-9a24-1ea2eb418d9d admin admin] PATH_INFO: `/v3/users/a7a959e57e0c4713a855844645fdc26a` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:51:46.003465 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2645d4b-d6e6-4904-9a24-1ea2eb418d9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7a959e57e0c4713a855844645fdc26a)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:51:46.005436 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2645d4b-d6e6-4904-9a24-1ea2eb418d9d admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:51:46.067892 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c2645d4b-d6e6-4904-9a24-1ea2eb418d9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7a959e57e0c4713a855844645fdc26a'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:46.068566 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c2645d4b-d6e6-4904-9a24-1ea2eb418d9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7a959e57e0c4713a855844645fdc26a'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:46.082477 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c2645d4b-d6e6-4904-9a24-1ea2eb418d9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7a959e57e0c4713a855844645fdc26a'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:46.093312 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c2645d4b-d6e6-4904-9a24-1ea2eb418d9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7a959e57e0c4713a855844645fdc26a'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:46.102191 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-c2645d4b-d6e6-4904-9a24-1ea2eb418d9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7a959e57e0c4713a855844645fdc26a'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:46.109494 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1462/2913] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:51:45 2022] DELETE /identity/v3/users/a7a959e57e0c4713a855844645fdc26a => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:51:46.297621 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81c2593-02e3-410d-997b-ef04adf48034 service swift] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:51:46.308759 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81c2593-02e3-410d-997b-ef04adf48034 service swift] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:51:46.371258 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54db1af9-973d-451a-9427-ee74e198c980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:51:46.371920 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db1af9-973d-451a-9427-ee74e198c980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:51:46.372203 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db1af9-973d-451a-9427-ee74e198c980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:51:46.372546 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db1af9-973d-451a-9427-ee74e198c980 admin admin] PATH_INFO: `/v3/projects/069626c3525d47a9b67f1e6d81b47b2a` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:51:46.373718 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54db1af9-973d-451a-9427-ee74e198c980 admin admin] RBAC: Authorizing `identity:delete_project(project_id=069626c3525d47a9b67f1e6d81b47b2a)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:51:46.375694 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54db1af9-973d-451a-9427-ee74e198c980 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:51:46.426948 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-54db1af9-973d-451a-9427-ee74e198c980 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '069626c3525d47a9b67f1e6d81b47b2a', 'request_id': 'req-54db1af9-973d-451a-9427-ee74e198c980'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:46.435463 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-54db1af9-973d-451a-9427-ee74e198c980 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '069626c3525d47a9b67f1e6d81b47b2a', 'request_id': 'req-54db1af9-973d-451a-9427-ee74e198c980'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:46.442111 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-54db1af9-973d-451a-9427-ee74e198c980 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '069626c3525d47a9b67f1e6d81b47b2a', 'request_id': 'req-54db1af9-973d-451a-9427-ee74e198c980'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:46.446158 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:51:46.456457 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-54db1af9-973d-451a-9427-ee74e198c980 admin admin] The token cache is being invalidate because project 069626c3525d47a9b67f1e6d81b47b2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:51:46.456977 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-54db1af9-973d-451a-9427-ee74e198c980 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:46.458260 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-54db1af9-973d-451a-9427-ee74e198c980 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:46.460008 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1452/2914] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:51:46 2022] DELETE /identity/v3/projects/069626c3525d47a9b67f1e6d81b47b2a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:51:46.464694 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2645d4b-d6e6-4904-9a24-1ea2eb418d9d admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:51:46.466163 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-c2645d4b-d6e6-4904-9a24-1ea2eb418d9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:51:46.479200 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2645d4b-d6e6-4904-9a24-1ea2eb418d9d admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:51:46.539419 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b1c657-fd91-4064-9c01-55663bd11b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:51:46.539967 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1c657-fd91-4064-9c01-55663bd11b8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:51:46.540244 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1c657-fd91-4064-9c01-55663bd11b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:51:46.540511 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1c657-fd91-4064-9c01-55663bd11b8e admin admin] PATH_INFO: `/v3/users/2147ccd32c5b45d8bcea965ff5c5a849` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:51:46.541630 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b1c657-fd91-4064-9c01-55663bd11b8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2147ccd32c5b45d8bcea965ff5c5a849)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:51:46.543584 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b1c657-fd91-4064-9c01-55663bd11b8e admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:51:46.590905 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a2b1c657-fd91-4064-9c01-55663bd11b8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2147ccd32c5b45d8bcea965ff5c5a849'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:46.591547 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a2b1c657-fd91-4064-9c01-55663bd11b8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2147ccd32c5b45d8bcea965ff5c5a849'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:46.596887 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:51:46.604912 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a2b1c657-fd91-4064-9c01-55663bd11b8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2147ccd32c5b45d8bcea965ff5c5a849'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:46.612614 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a2b1c657-fd91-4064-9c01-55663bd11b8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2147ccd32c5b45d8bcea965ff5c5a849'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:46.621554 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-a2b1c657-fd91-4064-9c01-55663bd11b8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2147ccd32c5b45d8bcea965ff5c5a849'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:46.627932 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1463/2915] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:51:46 2022] DELETE /identity/v3/users/2147ccd32c5b45d8bcea965ff5c5a849 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:51:46.812818 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54db1af9-973d-451a-9427-ee74e198c980 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:51:46.814294 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-54db1af9-973d-451a-9427-ee74e198c980 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:51:46.829431 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54db1af9-973d-451a-9427-ee74e198c980 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:51:46.888483 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaabf83-c259-42cc-9288-731406b44cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:51:46.889095 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaabf83-c259-42cc-9288-731406b44cc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:51:46.889403 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaabf83-c259-42cc-9288-731406b44cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:51:46.889676 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaabf83-c259-42cc-9288-731406b44cc1 admin admin] PATH_INFO: `/v3/projects/50b7a9ed7d494df39cc0040c1ca761b8` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:51:46.890816 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaabf83-c259-42cc-9288-731406b44cc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50b7a9ed7d494df39cc0040c1ca761b8)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:51:46.892737 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaabf83-c259-42cc-9288-731406b44cc1 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:51:46.938731 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-beaabf83-c259-42cc-9288-731406b44cc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50b7a9ed7d494df39cc0040c1ca761b8', 'request_id': 'req-beaabf83-c259-42cc-9288-731406b44cc1'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:46.946238 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-beaabf83-c259-42cc-9288-731406b44cc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50b7a9ed7d494df39cc0040c1ca761b8', 'request_id': 'req-beaabf83-c259-42cc-9288-731406b44cc1'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:46.952936 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-beaabf83-c259-42cc-9288-731406b44cc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50b7a9ed7d494df39cc0040c1ca761b8', 'request_id': 'req-beaabf83-c259-42cc-9288-731406b44cc1'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:46.963065 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-beaabf83-c259-42cc-9288-731406b44cc1 admin admin] The token cache is being invalidate because project 50b7a9ed7d494df39cc0040c1ca761b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:51:46.963351 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-beaabf83-c259-42cc-9288-731406b44cc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:46.963978 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-beaabf83-c259-42cc-9288-731406b44cc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:51:46.965453 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1453/2916] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:51:46 2022] DELETE /identity/v3/projects/50b7a9ed7d494df39cc0040c1ca761b8 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:51:56.624609 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=10,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:51:56.960136 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:DELETE=8,keystone:SELECT=53,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:52:14.810798 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b1c657-fd91-4064-9c01-55663bd11b8e admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:52:14.812573 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a2b1c657-fd91-4064-9c01-55663bd11b8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:52:14.927530 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b1c657-fd91-4064-9c01-55663bd11b8e admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:52:14.999214 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca51933a-1816-46a5-9d10-597ec2d3b3f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf14b64bf04b2ba74d2fe1fd41ff71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:52:14.999849 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51933a-1816-46a5-9d10-597ec2d3b3f0 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:52:15.000128 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51933a-1816-46a5-9d10-597ec2d3b3f0 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:52:15.000396 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51933a-1816-46a5-9d10-597ec2d3b3f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:52:15.002423 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ca51933a-1816-46a5-9d10-597ec2d3b3f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:52:15.015951 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca51933a-1816-46a5-9d10-597ec2d3b3f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:52:15.017571 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca51933a-1816-46a5-9d10-597ec2d3b3f0 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:52:15.030301 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1464/2917] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:52:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:52:15.941168 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355dffba-aaea-4672-b179-9d09dd8517a3 None None] REQUEST_METHOD: `POST` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:52:15.941732 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355dffba-aaea-4672-b179-9d09dd8517a3 None None] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:52:15.941732 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355dffba-aaea-4672-b179-9d09dd8517a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:52:15.971509 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.auth.core [None req-355dffba-aaea-4672-b179-9d09dd8517a3 None None] MFA Rules not processed for user `5fac9ab7665b4f14936adea96ae22ec4`. Rule list: `[]` (Enabled: `True`). {{(pid=81604) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:52:15.973577 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-355dffba-aaea-4672-b179-9d09dd8517a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:52:16.055673 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1454/2918] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:52:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:52:16.848138 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca51933a-1816-46a5-9d10-597ec2d3b3f0 service nova] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:52:16.849685 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ca51933a-1816-46a5-9d10-597ec2d3b3f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:52:16.922083 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca51933a-1816-46a5-9d10-597ec2d3b3f0 service nova] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:52:16.927018 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af15f852-5299-449d-bc2e-859a96139ac1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de934d0105bd4d01ba6c06a4a0d36704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:52:16.927620 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15f852-5299-449d-bc2e-859a96139ac1 service placement] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:52:16.927898 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15f852-5299-449d-bc2e-859a96139ac1 service placement] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:52:16.928169 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15f852-5299-449d-bc2e-859a96139ac1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:52:16.929924 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-af15f852-5299-449d-bc2e-859a96139ac1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:52:16.942131 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af15f852-5299-449d-bc2e-859a96139ac1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:52:16.943700 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af15f852-5299-449d-bc2e-859a96139ac1 service placement] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:52:16.954308 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1465/2919] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:52:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:52:19.008484 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355dffba-aaea-4672-b179-9d09dd8517a3 None None] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:52:19.010452 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-355dffba-aaea-4672-b179-9d09dd8517a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:52:19.084358 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355dffba-aaea-4672-b179-9d09dd8517a3 None None] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:52:19.090860 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded80de5-8694-4a31-aa7b-d3df9b11a0d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:52:19.091816 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded80de5-8694-4a31-aa7b-d3df9b11a0d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:52:19.092600 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded80de5-8694-4a31-aa7b-d3df9b11a0d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:52:19.093056 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded80de5-8694-4a31-aa7b-d3df9b11a0d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:52:19.096200 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ded80de5-8694-4a31-aa7b-d3df9b11a0d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:52:19.116188 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded80de5-8694-4a31-aa7b-d3df9b11a0d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:52:19.118886 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded80de5-8694-4a31-aa7b-d3df9b11a0d5 service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:52:19.133326 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1455/2920] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:52:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:52:26.950006 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=94 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:52:29.127115 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=51 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:52:34.352514 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af15f852-5299-449d-bc2e-859a96139ac1 service placement] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:52:34.354104 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-af15f852-5299-449d-bc2e-859a96139ac1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:52:34.369503 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af15f852-5299-449d-bc2e-859a96139ac1 service placement] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:52:34.375373 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1aa32d-d577-41ed-837b-656d9d22e867 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf14b64bf04b2ba74d2fe1fd41ff71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:52:34.376008 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1aa32d-d577-41ed-837b-656d9d22e867 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:52:34.376265 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1aa32d-d577-41ed-837b-656d9d22e867 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:52:34.376537 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1aa32d-d577-41ed-837b-656d9d22e867 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:52:34.378307 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ea1aa32d-d577-41ed-837b-656d9d22e867 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:52:34.392626 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1aa32d-d577-41ed-837b-656d9d22e867 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:52:34.394313 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1aa32d-d577-41ed-837b-656d9d22e867 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:52:34.407249 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1466/2921] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:52:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:52:44.402393 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=6 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:53:13.943848 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded80de5-8694-4a31-aa7b-d3df9b11a0d5 service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:53:13.947893 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-ded80de5-8694-4a31-aa7b-d3df9b11a0d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:53:13.978799 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded80de5-8694-4a31-aa7b-d3df9b11a0d5 service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:53:13.984762 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1847315-2ea6-45c2-9177-69c32cc3921e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de934d0105bd4d01ba6c06a4a0d36704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:53:13.985304 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1847315-2ea6-45c2-9177-69c32cc3921e service placement] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:53:13.985581 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1847315-2ea6-45c2-9177-69c32cc3921e service placement] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:53:13.985869 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1847315-2ea6-45c2-9177-69c32cc3921e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:53:13.988289 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-d1847315-2ea6-45c2-9177-69c32cc3921e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:53:14.006316 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1847315-2ea6-45c2-9177-69c32cc3921e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:53:14.007930 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1847315-2ea6-45c2-9177-69c32cc3921e service placement] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:53:14.020670 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1456/2922] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:53:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:53:17.115112 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1aa32d-d577-41ed-837b-656d9d22e867 service nova] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:53:17.116940 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-ea1aa32d-d577-41ed-837b-656d9d22e867 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:53:17.190231 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1aa32d-d577-41ed-837b-656d9d22e867 service nova] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:53:17.194724 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78510c-354e-4501-860a-9b1d3bcda5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:53:17.195338 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78510c-354e-4501-860a-9b1d3bcda5f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:53:17.195615 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78510c-354e-4501-860a-9b1d3bcda5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:53:17.195886 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78510c-354e-4501-860a-9b1d3bcda5f7 admin admin] PATH_INFO: `/v3/users/55a6ab5102034cc68a60218d0d434f25` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:53:17.196981 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b78510c-354e-4501-860a-9b1d3bcda5f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55a6ab5102034cc68a60218d0d434f25)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:53:17.198922 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b78510c-354e-4501-860a-9b1d3bcda5f7 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:53:17.244859 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9b78510c-354e-4501-860a-9b1d3bcda5f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55a6ab5102034cc68a60218d0d434f25'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:53:17.245496 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9b78510c-354e-4501-860a-9b1d3bcda5f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55a6ab5102034cc68a60218d0d434f25'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:53:17.257119 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9b78510c-354e-4501-860a-9b1d3bcda5f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55a6ab5102034cc68a60218d0d434f25'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:53:17.264545 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9b78510c-354e-4501-860a-9b1d3bcda5f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55a6ab5102034cc68a60218d0d434f25'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:53:17.271657 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-9b78510c-354e-4501-860a-9b1d3bcda5f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55a6ab5102034cc68a60218d0d434f25'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:53:17.279666 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1467/2923] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:53:17 2022] DELETE /identity/v3/users/55a6ab5102034cc68a60218d0d434f25 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:53:17.464071 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1847315-2ea6-45c2-9177-69c32cc3921e service placement] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:53:17.484119 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1847315-2ea6-45c2-9177-69c32cc3921e service placement] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:53:17.551491 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95989be6-4511-41aa-8ba7-4fae531b2a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:53:17.552128 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95989be6-4511-41aa-8ba7-4fae531b2a5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:53:17.552411 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95989be6-4511-41aa-8ba7-4fae531b2a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:53:17.552681 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95989be6-4511-41aa-8ba7-4fae531b2a5b admin admin] PATH_INFO: `/v3/projects/ed1bf756bfcb4d52bd82979559f03a20` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:53:17.553789 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95989be6-4511-41aa-8ba7-4fae531b2a5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed1bf756bfcb4d52bd82979559f03a20)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:53:17.555718 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95989be6-4511-41aa-8ba7-4fae531b2a5b admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:53:17.606077 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-95989be6-4511-41aa-8ba7-4fae531b2a5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed1bf756bfcb4d52bd82979559f03a20', 'request_id': 'req-95989be6-4511-41aa-8ba7-4fae531b2a5b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:53:17.613494 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-95989be6-4511-41aa-8ba7-4fae531b2a5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed1bf756bfcb4d52bd82979559f03a20', 'request_id': 'req-95989be6-4511-41aa-8ba7-4fae531b2a5b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:53:17.620000 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-95989be6-4511-41aa-8ba7-4fae531b2a5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed1bf756bfcb4d52bd82979559f03a20', 'request_id': 'req-95989be6-4511-41aa-8ba7-4fae531b2a5b'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:53:17.630287 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-95989be6-4511-41aa-8ba7-4fae531b2a5b admin admin] The token cache is being invalidate because project ed1bf756bfcb4d52bd82979559f03a20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:53:17.630575 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-95989be6-4511-41aa-8ba7-4fae531b2a5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:53:17.631198 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-95989be6-4511-41aa-8ba7-4fae531b2a5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:53:17.632691 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1457/2924] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:53:17 2022] DELETE /identity/v3/projects/ed1bf756bfcb4d52bd82979559f03a20 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:53:17.636689 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78510c-354e-4501-860a-9b1d3bcda5f7 admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:53:17.638153 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-9b78510c-354e-4501-860a-9b1d3bcda5f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:53:17.651466 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78510c-354e-4501-860a-9b1d3bcda5f7 admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:53:17.706154 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe64b7-b240-4fd3-84c7-592800d4ee80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:53:17.706777 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe64b7-b240-4fd3-84c7-592800d4ee80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:53:17.707225 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe64b7-b240-4fd3-84c7-592800d4ee80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:53:17.707353 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe64b7-b240-4fd3-84c7-592800d4ee80 admin admin] PATH_INFO: `/v3/users/87b8824b42a942b4a7b83d1a10f752ad` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:53:17.709022 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fe64b7-b240-4fd3-84c7-592800d4ee80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87b8824b42a942b4a7b83d1a10f752ad)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:53:17.710991 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fe64b7-b240-4fd3-84c7-592800d4ee80 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:53:17.732477 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:53:17.759806 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-72fe64b7-b240-4fd3-84c7-592800d4ee80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87b8824b42a942b4a7b83d1a10f752ad'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:53:17.760487 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-72fe64b7-b240-4fd3-84c7-592800d4ee80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87b8824b42a942b4a7b83d1a10f752ad'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:53:17.773056 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-72fe64b7-b240-4fd3-84c7-592800d4ee80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87b8824b42a942b4a7b83d1a10f752ad'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:53:17.780285 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-72fe64b7-b240-4fd3-84c7-592800d4ee80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87b8824b42a942b4a7b83d1a10f752ad'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:53:17.787386 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-72fe64b7-b240-4fd3-84c7-592800d4ee80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87b8824b42a942b4a7b83d1a10f752ad'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:53:17.793156 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1468/2925] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:53:17 2022] DELETE /identity/v3/users/87b8824b42a942b4a7b83d1a10f752ad => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:53:17.966018 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95989be6-4511-41aa-8ba7-4fae531b2a5b admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:53:17.967460 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-95989be6-4511-41aa-8ba7-4fae531b2a5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:53:17.980548 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95989be6-4511-41aa-8ba7-4fae531b2a5b admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:53:18.038475 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15295005-6b4c-42f0-9606-99b1d6c98f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:53:18.039102 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15295005-6b4c-42f0-9606-99b1d6c98f56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:53:18.039380 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15295005-6b4c-42f0-9606-99b1d6c98f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:53:18.039654 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15295005-6b4c-42f0-9606-99b1d6c98f56 admin admin] PATH_INFO: `/v3/projects/dec029c8807c43d6982f9e7fe23c90b2` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:53:18.040758 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15295005-6b4c-42f0-9606-99b1d6c98f56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dec029c8807c43d6982f9e7fe23c90b2)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:53:18.042693 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15295005-6b4c-42f0-9606-99b1d6c98f56 admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:53:18.063214 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:53:18.091061 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-15295005-6b4c-42f0-9606-99b1d6c98f56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dec029c8807c43d6982f9e7fe23c90b2', 'request_id': 'req-15295005-6b4c-42f0-9606-99b1d6c98f56'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:53:18.098220 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-15295005-6b4c-42f0-9606-99b1d6c98f56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dec029c8807c43d6982f9e7fe23c90b2', 'request_id': 'req-15295005-6b4c-42f0-9606-99b1d6c98f56'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:53:18.104607 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-15295005-6b4c-42f0-9606-99b1d6c98f56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dec029c8807c43d6982f9e7fe23c90b2', 'request_id': 'req-15295005-6b4c-42f0-9606-99b1d6c98f56'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:53:18.115788 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-15295005-6b4c-42f0-9606-99b1d6c98f56 admin admin] The token cache is being invalidate because project dec029c8807c43d6982f9e7fe23c90b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81604) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:53:18.116072 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-15295005-6b4c-42f0-9606-99b1d6c98f56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:53:18.116700 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-15295005-6b4c-42f0-9606-99b1d6c98f56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:53:18.118195 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1458/2926] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:53:17 2022] DELETE /identity/v3/projects/dec029c8807c43d6982f9e7fe23c90b2 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:53:27.789817 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:DELETE=7,keystone:SELECT=20,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:53:28.112277 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:54:11.139906 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f963b0-d4d2-49b6-9c4f-1af2feb6856b None None] REQUEST_METHOD: `POST` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:54:11.140448 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f963b0-d4d2-49b6-9c4f-1af2feb6856b None None] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:54:11.140448 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f963b0-d4d2-49b6-9c4f-1af2feb6856b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:54:11.159929 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.auth.core [None req-19f963b0-d4d2-49b6-9c4f-1af2feb6856b None None] MFA Rules not processed for user `5fac9ab7665b4f14936adea96ae22ec4`. Rule list: `[]` (Enabled: `True`). {{(pid=81605) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 28 15:54:11.161108 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-19f963b0-d4d2-49b6-9c4f-1af2feb6856b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:54:11.214230 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1469/2927] 10.222.0.52 () {58 vars in 958 bytes} [Wed Sep 28 15:54:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 28 15:54:11.542201 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15295005-6b4c-42f0-9606-99b1d6c98f56 admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:54:11.543520 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-15295005-6b4c-42f0-9606-99b1d6c98f56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:54:11.599907 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15295005-6b4c-42f0-9606-99b1d6c98f56 admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:54:11.603760 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c56a2ed-18c8-4d50-b92e-cc8d4739b3db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:54:11.604357 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c56a2ed-18c8-4d50-b92e-cc8d4739b3db service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:54:11.604632 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c56a2ed-18c8-4d50-b92e-cc8d4739b3db service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:54:11.604902 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c56a2ed-18c8-4d50-b92e-cc8d4739b3db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:54:11.606626 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7c56a2ed-18c8-4d50-b92e-cc8d4739b3db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:54:11.616733 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c56a2ed-18c8-4d50-b92e-cc8d4739b3db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:54:11.618168 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c56a2ed-18c8-4d50-b92e-cc8d4739b3db service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:54:11.626682 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1459/2928] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:54:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:54:21.207744 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=25 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:54:21.622533 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=26 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:54:25.033569 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f963b0-d4d2-49b6-9c4f-1af2feb6856b None None] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:54:25.035062 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-19f963b0-d4d2-49b6-9c4f-1af2feb6856b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:54:25.121877 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f963b0-d4d2-49b6-9c4f-1af2feb6856b None None] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:54:25.127471 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab4129a-3689-4452-8e5b-c325f0f79078 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8daf14b64bf04b2ba74d2fe1fd41ff71', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:54:25.128080 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab4129a-3689-4452-8e5b-c325f0f79078 service nova] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:54:25.128357 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab4129a-3689-4452-8e5b-c325f0f79078 service nova] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:54:25.128626 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab4129a-3689-4452-8e5b-c325f0f79078 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:54:25.130524 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-3ab4129a-3689-4452-8e5b-c325f0f79078 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:54:25.144065 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab4129a-3689-4452-8e5b-c325f0f79078 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:54:25.145649 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab4129a-3689-4452-8e5b-c325f0f79078 service nova] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:54:25.157136 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1470/2929] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:54:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:54:35.151977 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=34 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:55:41.687600 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c56a2ed-18c8-4d50-b92e-cc8d4739b3db service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:55:41.689712 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-7c56a2ed-18c8-4d50-b92e-cc8d4739b3db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:55:41.759109 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c56a2ed-18c8-4d50-b92e-cc8d4739b3db service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:55:41.763974 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b28012-d3c1-4b16-b64b-b337999b8b8c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de934d0105bd4d01ba6c06a4a0d36704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:55:41.764565 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b28012-d3c1-4b16-b64b-b337999b8b8c service placement] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:55:41.764837 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b28012-d3c1-4b16-b64b-b337999b8b8c service placement] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:55:41.765099 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b28012-d3c1-4b16-b64b-b337999b8b8c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:55:41.766787 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-e5b28012-d3c1-4b16-b64b-b337999b8b8c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:55:41.778214 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b28012-d3c1-4b16-b64b-b337999b8b8c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:55:41.779794 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b28012-d3c1-4b16-b64b-b337999b8b8c service placement] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:55:41.792746 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1460/2930] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:55:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:55:51.787490 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=32 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:55:58.976054 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab4129a-3689-4452-8e5b-c325f0f79078 service nova] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:55:58.985478 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab4129a-3689-4452-8e5b-c325f0f79078 service nova] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:55:58.989178 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a51f37e-0b78-409b-afe8-9c000c54d49d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:55:58.989778 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a51f37e-0b78-409b-afe8-9c000c54d49d service neutron] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:55:58.990091 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a51f37e-0b78-409b-afe8-9c000c54d49d service neutron] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:55:58.990364 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a51f37e-0b78-409b-afe8-9c000c54d49d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:55:58.992094 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6a51f37e-0b78-409b-afe8-9c000c54d49d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:55:59.003480 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a51f37e-0b78-409b-afe8-9c000c54d49d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:55:59.004928 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a51f37e-0b78-409b-afe8-9c000c54d49d service neutron] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:55:59.017248 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1471/2931] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:55:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:56:09.010333 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=6 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:56:12.652048 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b28012-d3c1-4b16-b64b-b337999b8b8c service placement] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:56:12.662033 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b28012-d3c1-4b16-b64b-b337999b8b8c service placement] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:56:12.666979 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5def3402-ca2f-4ee2-a5d2-918821b15f4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b13b0533444a449ead00a389c5f6bd1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:56:12.667681 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def3402-ca2f-4ee2-a5d2-918821b15f4f service neutron] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:56:12.667959 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def3402-ca2f-4ee2-a5d2-918821b15f4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:56:12.668259 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5def3402-ca2f-4ee2-a5d2-918821b15f4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:56:12.670187 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-5def3402-ca2f-4ee2-a5d2-918821b15f4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:56:12.741682 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5def3402-ca2f-4ee2-a5d2-918821b15f4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:56:12.743171 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5def3402-ca2f-4ee2-a5d2-918821b15f4f service neutron] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:56:12.753043 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1461/2932] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:56:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:56:16.084759 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a51f37e-0b78-409b-afe8-9c000c54d49d service neutron] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:56:16.086194 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-6a51f37e-0b78-409b-afe8-9c000c54d49d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:56:16.109422 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a51f37e-0b78-409b-afe8-9c000c54d49d service neutron] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:56:16.116807 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71e492d-8862-4aec-b79b-5e5f94b88698 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de934d0105bd4d01ba6c06a4a0d36704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:56:16.117414 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e492d-8862-4aec-b79b-5e5f94b88698 service placement] REQUEST_METHOD: `GET` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:56:16.117693 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e492d-8862-4aec-b79b-5e5f94b88698 service placement] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:56:16.118040 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71e492d-8862-4aec-b79b-5e5f94b88698 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:56:16.120218 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-a71e492d-8862-4aec-b79b-5e5f94b88698 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:56:16.133930 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71e492d-8862-4aec-b79b-5e5f94b88698 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:56:16.135517 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71e492d-8862-4aec-b79b-5e5f94b88698 service placement] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:56:16.147886 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1472/2933] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 15:56:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 15:56:19.211348 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5def3402-ca2f-4ee2-a5d2-918821b15f4f service neutron] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:56:19.213779 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-5def3402-ca2f-4ee2-a5d2-918821b15f4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:56:19.237646 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5def3402-ca2f-4ee2-a5d2-918821b15f4f service neutron] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:56:19.242325 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dcd015-9917-49fd-a70e-d9e39c108eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:56:19.242949 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dcd015-9917-49fd-a70e-d9e39c108eba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:56:19.243222 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dcd015-9917-49fd-a70e-d9e39c108eba admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:56:19.243491 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dcd015-9917-49fd-a70e-d9e39c108eba admin admin] PATH_INFO: `/v3/users/5fac9ab7665b4f14936adea96ae22ec4` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:56:19.244619 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dcd015-9917-49fd-a70e-d9e39c108eba admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fac9ab7665b4f14936adea96ae22ec4)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:56:19.246561 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dcd015-9917-49fd-a70e-d9e39c108eba admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:56:19.303675 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-53dcd015-9917-49fd-a70e-d9e39c108eba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fac9ab7665b4f14936adea96ae22ec4'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:56:19.313479 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-53dcd015-9917-49fd-a70e-d9e39c108eba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fac9ab7665b4f14936adea96ae22ec4'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:56:19.314200 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-53dcd015-9917-49fd-a70e-d9e39c108eba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fac9ab7665b4f14936adea96ae22ec4'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:56:19.322413 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-53dcd015-9917-49fd-a70e-d9e39c108eba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fac9ab7665b4f14936adea96ae22ec4'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:56:19.323062 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-53dcd015-9917-49fd-a70e-d9e39c108eba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fac9ab7665b4f14936adea96ae22ec4'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:56:19.346361 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1462/2934] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:56:19 2022] DELETE /identity/v3/users/5fac9ab7665b4f14936adea96ae22ec4 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 28 15:56:19.528358 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71e492d-8862-4aec-b79b-5e5f94b88698 service placement] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:56:19.541737 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71e492d-8862-4aec-b79b-5e5f94b88698 service placement] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:56:19.607883 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4404a2fa-c0a7-476a-a9ea-ad979bc99ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:56:19.608609 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4404a2fa-c0a7-476a-a9ea-ad979bc99ddb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:56:19.608962 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4404a2fa-c0a7-476a-a9ea-ad979bc99ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:56:19.609302 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4404a2fa-c0a7-476a-a9ea-ad979bc99ddb admin admin] PATH_INFO: `/v3/projects/35cf5a9c74e44a918e45867c6570aa11` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:56:19.611015 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4404a2fa-c0a7-476a-a9ea-ad979bc99ddb admin admin] RBAC: Authorizing `identity:delete_project(project_id=35cf5a9c74e44a918e45867c6570aa11)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:56:19.613037 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4404a2fa-c0a7-476a-a9ea-ad979bc99ddb admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:56:19.667651 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4404a2fa-c0a7-476a-a9ea-ad979bc99ddb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35cf5a9c74e44a918e45867c6570aa11', 'request_id': 'req-4404a2fa-c0a7-476a-a9ea-ad979bc99ddb'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:56:19.674639 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4404a2fa-c0a7-476a-a9ea-ad979bc99ddb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35cf5a9c74e44a918e45867c6570aa11', 'request_id': 'req-4404a2fa-c0a7-476a-a9ea-ad979bc99ddb'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:56:19.684814 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4404a2fa-c0a7-476a-a9ea-ad979bc99ddb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35cf5a9c74e44a918e45867c6570aa11', 'request_id': 'req-4404a2fa-c0a7-476a-a9ea-ad979bc99ddb'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:56:19.697662 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4404a2fa-c0a7-476a-a9ea-ad979bc99ddb admin admin] The token cache is being invalidate because project 35cf5a9c74e44a918e45867c6570aa11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:56:19.697996 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4404a2fa-c0a7-476a-a9ea-ad979bc99ddb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:56:19.699304 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-4404a2fa-c0a7-476a-a9ea-ad979bc99ddb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:56:19.702140 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1473/2935] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:56:19 2022] DELETE /identity/v3/projects/35cf5a9c74e44a918e45867c6570aa11 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:56:19.706363 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dcd015-9917-49fd-a70e-d9e39c108eba admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:56:19.707832 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-53dcd015-9917-49fd-a70e-d9e39c108eba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:56:19.725204 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dcd015-9917-49fd-a70e-d9e39c108eba admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:56:19.786711 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43632661-c339-46aa-9da7-757d573aa5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:56:19.787343 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43632661-c339-46aa-9da7-757d573aa5aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:56:19.787613 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43632661-c339-46aa-9da7-757d573aa5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:56:19.787881 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43632661-c339-46aa-9da7-757d573aa5aa admin admin] PATH_INFO: `/v3/users/021ed1577ed84746b17a68da8ffd6bea` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:56:19.789033 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43632661-c339-46aa-9da7-757d573aa5aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=021ed1577ed84746b17a68da8ffd6bea)` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:56:19.791355 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43632661-c339-46aa-9da7-757d573aa5aa admin admin] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:56:19.797380 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:56:19.846277 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-43632661-c339-46aa-9da7-757d573aa5aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '021ed1577ed84746b17a68da8ffd6bea'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:56:19.853374 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-43632661-c339-46aa-9da7-757d573aa5aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '021ed1577ed84746b17a68da8ffd6bea'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:56:19.854099 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-43632661-c339-46aa-9da7-757d573aa5aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '021ed1577ed84746b17a68da8ffd6bea'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:56:19.863287 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-43632661-c339-46aa-9da7-757d573aa5aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '021ed1577ed84746b17a68da8ffd6bea'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:56:19.863925 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.notifications [None req-43632661-c339-46aa-9da7-757d573aa5aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '021ed1577ed84746b17a68da8ffd6bea'} {{(pid=81604) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:56:19.881771 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1463/2936] 10.222.0.52 () {58 vars in 1224 bytes} [Wed Sep 28 15:56:19 2022] DELETE /identity/v3/users/021ed1577ed84746b17a68da8ffd6bea => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:56:20.062050 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4404a2fa-c0a7-476a-a9ea-ad979bc99ddb admin admin] Authenticating user token {{(pid=81605) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 15:56:20.063501 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.fernet_utils [None req-4404a2fa-c0a7-476a-a9ea-ad979bc99ddb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81605) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 15:56:20.078615 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4404a2fa-c0a7-476a-a9ea-ad979bc99ddb admin admin] Validating token access rules against request {{(pid=81605) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 15:56:20.143872 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2ef1f-61c9-4b18-a6fd-abf52abd58c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fdbcf7ba1534603b18de5dbaf96574f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8de950cb25d34fe8afed2528d2e112e6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81605) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 15:56:20.144502 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2ef1f-61c9-4b18-a6fd-abf52abd58c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 15:56:20.144776 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2ef1f-61c9-4b18-a6fd-abf52abd58c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 15:56:20.145050 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2ef1f-61c9-4b18-a6fd-abf52abd58c1 admin admin] PATH_INFO: `/v3/projects/9f00f264edf140dba0b727a459c9aada` {{(pid=81605) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 15:56:20.146371 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f2ef1f-61c9-4b18-a6fd-abf52abd58c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f00f264edf140dba0b727a459c9aada)` {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 15:56:20.148360 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f2ef1f-61c9-4b18-a6fd-abf52abd58c1 admin admin] RBAC: Authorization granted {{(pid=81605) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 15:56:20.170242 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:56:20.198869 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e1f2ef1f-61c9-4b18-a6fd-abf52abd58c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f00f264edf140dba0b727a459c9aada', 'request_id': 'req-e1f2ef1f-61c9-4b18-a6fd-abf52abd58c1'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:56:20.204962 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e1f2ef1f-61c9-4b18-a6fd-abf52abd58c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f00f264edf140dba0b727a459c9aada', 'request_id': 'req-e1f2ef1f-61c9-4b18-a6fd-abf52abd58c1'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:56:20.213081 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e1f2ef1f-61c9-4b18-a6fd-abf52abd58c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f00f264edf140dba0b727a459c9aada', 'request_id': 'req-e1f2ef1f-61c9-4b18-a6fd-abf52abd58c1'} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:56:20.224565 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e1f2ef1f-61c9-4b18-a6fd-abf52abd58c1 admin admin] The token cache is being invalidate because project 9f00f264edf140dba0b727a459c9aada was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81605) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 28 15:56:20.224848 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e1f2ef1f-61c9-4b18-a6fd-abf52abd58c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:56:20.225481 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG keystone.notifications [None req-e1f2ef1f-61c9-4b18-a6fd-abf52abd58c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81605) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 28 15:56:20.226982 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: [pid: 81605|app: 0|req: 1474/2937] 10.222.0.52 () {58 vars in 1233 bytes} [Wed Sep 28 15:56:20 2022] DELETE /identity/v3/projects/9f00f264edf140dba0b727a459c9aada => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 28 15:56:29.878226 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 15:56:30.221207 dv-cinder-859319-2-iscsi devstack@keystone.service[81605]: DEBUG dbcounter [-] [81605] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81605) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 28 16:00:48.596103 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43632661-c339-46aa-9da7-757d573aa5aa admin admin] Authenticating user token {{(pid=81604) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 28 16:00:48.597492 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-43632661-c339-46aa-9da7-757d573aa5aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:00:48.746011 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43632661-c339-46aa-9da7-757d573aa5aa admin admin] Validating token access rules against request {{(pid=81604) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 28 16:00:48.805083 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0fec89-c343-483a-aac9-180e9645c77b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de934d0105bd4d01ba6c06a4a0d36704', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3dc6c5f5fcd34922b0d09cd2aaed6cf1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81604) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 28 16:00:48.805701 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0fec89-c343-483a-aac9-180e9645c77b service placement] REQUEST_METHOD: `GET` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 28 16:00:48.806013 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0fec89-c343-483a-aac9-180e9645c77b service placement] SCRIPT_NAME: `/identity` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 28 16:00:48.806284 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0fec89-c343-483a-aac9-180e9645c77b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81604) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 28 16:00:48.808007 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.fernet_utils [None req-9f0fec89-c343-483a-aac9-180e9645c77b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81604) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 28 16:00:48.820082 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0fec89-c343-483a-aac9-180e9645c77b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 28 16:00:48.821580 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0fec89-c343-483a-aac9-180e9645c77b service placement] RBAC: Authorization granted {{(pid=81604) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 28 16:00:48.831634 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: [pid: 81604|app: 0|req: 1464/2938] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Sep 28 16:00:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 28 16:00:58.827191 dv-cinder-859319-2-iscsi devstack@keystone.service[81604]: DEBUG dbcounter [-] [81604] Writing DB stats keystone:SELECT=89 {{(pid=81604) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}